Category Archives: Sciences

Haptic Memories

Of Pixels and Voxels and nervous messes

In 1995 I was employed in exhibit development at Discovery World, a museum of science, economics and technology. My job led to my involvement in several vastly different technologies and scientific fields, from hydraulics and lasers to electricity and health. One particularly interesting piece of technology with which I became involved was a “Haptic” interface, called “The Phantom.” Haptic, from the Greek, means touch, and the Phantom was intended to provide the user with virtual sense of touch via a single finger tip.

It looked much like a miniature architect’s lamp, an arm with several degrees of freedom, terminating in a thimble-like cup at the end, which, in turn, was attached to an armature governed by priceless little motors and sensors. The entire design intended to allow the user to move their finger as freely in space as any of their other digits, until they encountered a virtual obstacle. This might be something as simple as a simulated piece of paper, or sandpaper, or perhaps something more complex, a billiard ball, or banana, a wrist, or a wrist with a pulse.

Via the thimble, the controlling computer system could convey texture, viscosity, pressure, vibrations, movement — the entire range of things we can feel with our fingers, albeit not heat nor cold nor the pin-prick of pain. But one might pluck an invisible guitar string, and feel its harmonics, or palpate the back of a virtual patient.

My group were unsure just what we would have the device simulate, nor how we would allow a visiting public to interact with it, given the inherent fragility of the device (and the largely reckless tendencies of the public). But as this was very new technology, having just been invented a year earlier by an MIT grad student, there was a scholarly conference about it, held near MIT, in suburban Boston, and I was to attend. In fact, when I received my conference credentials I was pleasantly bemused to see that I was credited, on MIT stationary, as Doctor Nic Bernstein. Doctor indeed!

Upon arrival at the conference assembly I was greeted by a curious assortment of engineers, scientists, investigators, doctors, physicists. Oh, and a three-star General from the US Army; Defense Advanced Research Projects Agency (DARPA), the same people who invented the Internet. Other than myself, and a geologist from Australia, everyone else there was, in some way or another, in the pocket of this General, something I pointed out during our plenary introductions. At the next meal break, said General sought me out as a dining companion. How, he needed to know, was I not also on his payroll?

There were several obvious implementations of the Phantom being discussed, such as a medical school using them, along with surgical dummies, to help physicians learn proper technique for administering epidermal injections & draws.

The Australian geologist was working with seismic stimulators to probe for deeply buried oil & gas deposits. This being done by amassing the vast amounts of three dimensional data produced by seismic stimulation — essentially carefully calibrated “shakers” attached by outrigger arms to long, low trucks, like massive insects, which would slowly advance along a grid work, shake the ground a bit, raise, advance some more, lower, shake, etc. Once a full grid had been worked, and the data assembled into a three dimensional model, the investigator would probe through the data, feeling his or her way along veins of ore or into voids filled with gas or oil; each substance represented with a different virtual viscosity.

During a field trip to the labs of the Mitsubishi Heavy Industry corporation, a friendly scientist showed me the system they were developing to help orthopaedists feel their way around (“appreciate” in the parlance) the knee joint of a prospective surgical subject, prior to wielding an actual knife.

Here’s how it was done. The patient would receive a scan — PET, CAT, MRI, whichever technology would best image the tissues involved — and the data would be loaded into a computer model. Rather than the pixels (Picture Elements) we think of from the two dimensional world of television or video, or printing, this data were rendered into Voxels, Volumetric Elements. In addition to the X, Y, & Z coordinates of a datum, there was also information on the density of the matter, rendered to the “viewer” as viscosity or resistance. A doctor could thus feel around the back side of a kneecap, for example, to appreciate the condition of the soft tissues there (if any remained), such as cartilage or muscle, each rendered in a different haptic manner.

It was fascinating. This was 1995 remember, long before these sort of things were depicted as routine in movies and on telly.

Software Memories

Postilion mailbox view

Postilion mailbox view

Many years ago, around 1996, I wanted a better MUA, Mail User Agent — What most people would call an email client, or mail reader.  I ended up writing my own, and it was called Postilion.  I haven’t done anything with this program since some time in 2001, but was surprised to wake up this morning to a message from a user.  Here is that message, and my reply.

As a note, “curses” is a library for manipulating standard old text displays (not graphical programs) and OpenBSD is an operating system similar to Linux.

Here, then, is the original message, from Mayuresh:

Hello,

I used to use “Postilion” under Linux running WindowMaker.
I found your application to be quite well designed on the UI front.
I migrated over to OpenBSD 5 years back and found it to be exactly what the team claims it to be, simple, clean and secure.

The OpenBSD team has just delivered a SMTP daemon;
http://www.openbsd.org/cgi-bin/man.cgi?query=smtpd&sektion=8

It would be nice to have a well designed MUA too.
Would you be interested in designing and developing a curses based MUA?

Best.

Here is my reply:

Mayuresh,
Thanks for your kind words about Postilion.  I haven’t had a message about that software in years.  I stopped working on it in 2001, and haven’t even used it myself for years.

I don’t think I’m the one to handle making a new (text based) curses MUA.  I am not a C programmer (Postilion was mostly TCL/Tk) and I don’t really have any interest in using a curses MUA.  That’s really key — I can’t see developing a software project that I myself am not interested in using.

When I first started work on Postilion, I had just read Eric Raymond’s (esr) “The Cathedral and the Bazaar“.  I was impressed by the Mail.app program available from NeXT, which I could run on NeXT/Step on a Sun system I had.  Mail.app, however, did not support IMAP, and had other limitations.  I was itching to have an MUA which matched the elegance of Mail.app but with more modern features.  This being in 1996, that meant IMAP, ACAP, multiple folders open at once, multiple identities, etc.

After reading Cathedral, I decided to take esr’s words to heart.  He talked about how important it was that the development projects a programmer took on had to scratch his own itch, and my itch was for an elegant MUA.  Secondly, he talked about finding a program which is close to what you want, and using that as a starting point.  I had already found TkRat, and figured that was a good place to start — the interface was written in TCL/Tk, an interpreted language (I am more comfortable with interpreted languages than with compiled ones) and the interface was fairly close to what I wanted.  Also, the author, Martin Forssén, was friendly and helpful, had a really good, robust C language core for handling the messy parts of the protocols (thanks to Mark Crispin’s fabulous c-client) and he seemed flattered that I was basing my work on his.  Our two projects gained a sort of symbioses over time.

Sure enough, as I worked on the project, I found that the Bazaar part of the process kicked into gear.  I soon had hundreds, then thousands of users.  My community was helpful, providing translations, testing, etc.  One user, Marco van Hylckama Vlieg, contributed the entire set of custom made icons used in the GUI.  At its peak, Postilion was is use on every continent (except, perhaps, Antarctica), in English, French, German, Italian, Dutch and Swedish.  It was favorably written up in a French computer journal and adopted as part of a civic computing infrastructure project in a German village.

I was honored by all of that.  It was exciting to get email messages from Abu Dabi or Moscow, Bucharest or Lima.  I sometimes got well over a hundred messages a day from people thanking me for my work, offering suggestions or requesting help.  I was hired by a call center management company to make custom additions to the core program, just to suit their environment.  My own employer at the time, an early ISP, paid me to add many of the features the software offered.

What made this all possible, however, was my own desire to use the software I was writing.  If you really want a good, curses based mail reader, Mayuresh, you should think about taking the journey I took!  Find something close to what you want, Mutt, say, or Alpine; something with a license you can abide; something in a language you’re either comfortable with, or feel you can learn, and have at it.  Take Eric Raymond’s advice — scratch your own itch, and start by building on the work of others.  I think you’ll find it a worthy mission.

I wish you well,
-nic

Just my little contribution to the Open Source world.  Not Postilion, that’s old news.  No, I mean prodding someone else to take the journey themselves.

London 2009 – Day 3 – The Neighbours

We already made reference to the fact that we are squatting right across the street from the headquarters of Saatchi & Saatchi, a worldwide advertising concern.  Next door to us, to the east, is an architectural firm, whose various models fill their windows (see photos).  Last night, on the way home from St. Martin’s, we noticed the name of the firm for the first time, it is none other than ARUP, one of the top design/build architectural engineering firms in the world.  There was a fantastic profile of on of ARUP head engineers in the New Yorker a year or so back.  They have been responsible for such projects as the Bird’s Nest stadium for the Beijing Olympics.

En Passant

Pawn has moved this past weekend, and just wants to share a few words about that.

Here they are:

Comet

That night. That cold crisp night that he watched the comet streak overhead. That night was the last that he could be said to have been responsible for his own actions. Not that he had exercised any great care in living his life up until this point. It’s just that in that strange and generous calculus which we apply to the decision making powers of the artistic class, he had been cut a lot of slack. Up until the night that comet cut a gash in the night sky and everything changed.

She wasn’t with him then, not sharing his appreciation for late night walks in the less than safe neighborhood in which they dwelt. She was back in the flat starting another novel and finishing another bottle of merlot. That is how it was, in those days; she, his erstwhile muse, had no muse of her own save bottle and book, while he, numb and tired of losing her every night to those twins, he strode away each night to find some peace within.

There was no peace without, it was all traffic noise and loud conversation in the immigrant heavy district. It was a symphony in rare parts – the low hum of the sodium-vapor lights, the rich indecipherable patois emanating from the myriad open windows, the staccato rhythm of the tram wheels as they teased and taunted the edges of the cobblestone that still poked up in several sections of the aging pavement. On top of all of that was the static crackle of the power arcing from the overhead lines to the commutators of the trams themselves. A festival of sounds spanning a century converged in his little part of creation and drew him out of himself and away from the tempestuous storm which was brewing in the synapses of his drunken muse back home, back at the flat, steeping herself in cheap reds and that special sense of betrayal which age visits upon those whose ambition has been left behind.

The comet, he did not know, was early. He was no student of these things, of astronomy, nor did he have any special interest in the facts behind it. He knew only that as he walked east there was a smudgy line arcing across the sky which he could not recall having seen before. Comets are known for their punctuality, they are the timekeepers of the heavens, in the sense of the apito; that whistle blown to keep the Amazonian rivers of musicians in Carnivalé parade on tempo. Much as the leader toots the apito as he runs up and down the length of the bataria to keep all those drummers in sync, the comets race around the firmament keeping all of the celestial watches synchronized. Until that night.

All of the best minds in science agreed that comet Shinberg-Takie was not due until 21:13 Zulu Time on 3 February. Shinberg-Takie had other plans it seemed. He did not understand this, nor would he come to appreciate the peculiar effects it was to have on his life as he entered into the gravitational tug of the comet that night. It was 10:45 on the 2nd of February when he left for his stroll, and Shinberg-Takie was already making a show in the eastern sky.

At 6:35 that evening, the large dish at Arecibo, Puerto Rico, was trained towards the eastern heavens. It operated in concert with much smaller optical telescopes from Yerkes to Griffith Park and points all over the globe as astronomers and astrophysicists struggled to understand how their eagerly awaited guest could possibly have arrived a full day early. One young graduate student in Berkeley’s sleepy astronomy department was watching the screens that night and before anyone else had noticed, he was already aware of the odd pull of ST-2008. He could no longer be held accountable either. He was already looking eastward, and waiting.

It was 8:35 in Rio and the stout yet fearsome bataria leader could not find his apito. How, he worried, would his beloved bataria sound without the steadying rhythmic guidance of his apito? The light in the eastern sky barely even registered as he, too, entered into its metaphysical orbit.

Shinberg-Takie had captured three souls by 21:45 Zulu. They all looked to the east and waited.

Whither Evil


Miguel Helft over at The New York Times blogs in Bits today about a new connector application from Cemaphore Systems, called MailShadow for Google Apps, which allows migration from MS Exchange to gmail. Interesting concept, and he raises some clever uses in the article. What caught my eye, though, was this slam from his reader,Mark, against Google and their reputation:

Why does no one ask the question “why would I want to put my mail on google’s servers?” when they scan it, index it, score it and have such a poor record of protecting anyone’s privacy. Their reputation as “of the people” is stunningly inaccurate given their willingness to hand over records to any government requesting them. They are not the NY Times protecting anyone’s rights or privacy. Their reputation is one of democratization and being “of the people” but they are not “of or for the person.” And it is the person – each set of eyeballs – that they make their money on.

I am happy to keep my mail on my exchange server or any server other than a company with so much hubris, money and power and so little respect for individual rights and privacy. And no willingness to use their position to protect individual rights. Much like the current Supreme Court, they trample them in the self-interest of their business expansion into any nation, regardless of that nation’s policies regarding individual rights, privacy or due process, and regardless of how it violates or damages the individual person who is responsible for Google’s financial value. And that’s the problem – they think they are completely responsible and that any individual google user is not.
Bringing Outlook and Gmail Closer Together – Bits – Technology – New York Times Blog

This joins a recent assault on Google, whose actions more often belie their much vaunted “Don’t be evil” slogan.

Remembrance Of Two Pioneers


Two people, each a giant in his field, and true pioneers, both passed away recently. Pawn was deeply influenced by both. Joseph Weizenbaum, pioneer in artificial intelligence and skeptic of technology’s role in human affairs passed away on March 5th, and Gus Giordano, pioneer in jazz dance and an extraordinarily gifted correographer passed away on March 9th.

Here is an excerpt from the New York Times obituary of Weizenbaum:

Eliza, written while Mr. Weizenbaum was a professor at the Massachusetts Institute of Technology in 1964 and 1965 and named after Eliza Doolittle, who learned proper English in “Pygmalion” and “My Fair Lady,” was a groundbreaking experiment in the study of human interaction with machines.

The program made it possible for a person typing in plain English at a computer terminal to interact with a machine in a semblance of a normal conversation. To dispense with the need for a large real-world database of information, the software parodied the part of a Rogerian therapist, frequently reframing a client’s statements as questions.

In fact, the responsiveness of the conversation was an illusion, because Eliza was programmed simply to respond to certain key words and phrases. That would lead to wild non sequiturs and bizarre detours, but Mr. Weizenbaum later said that he was stunned to discover that his students and others became deeply engrossed in conversations with the program, occasionally revealing intimate personal details.
Joseph Weizenbaum, Famed Programmer, Is Dead at 85 – New York Times

A friend and mentor introduced me to Eliza in 1976, about a decade after its conception, and it opened my eyes to what could be done with what are now called human machine interface facilities (commonly referred to as UI). Much of my professional work with technology, whether in computer fields or in exhibit development have been influenced by those early lessons.

In 1980 I had the honor to work on several dance performances with Gus Giordano Dance Chicago, when they came to the humble Metropole Theater in Milwaukee where I did lighting and tech work at the time. Here is an excerpt from the Times’ obituary of Giordano:

Mr. Giordano was best known through the performing of his company, Giordano Jazz Dance Chicago, founded in 1962 and based in Evanston, and through his teaching at dance conventions throughout the United States.

The company, now directed by Nan Giordano, his daughter, is said to have been the first dance troupe to dedicate itself solely to jazz dance. The company’s programs featured pieces by Mr. Giordano and later, as he grew older, included dances by guest choreographers including Mia Michaels and Davis Robertson. The performers became known for their strong training, energy and hard-driving, precise way of moving.

“Their sleek lines and high, silent jumps had the feel of a well-oiled 1958 Chevrolet Impala, a pure expression of another era and something we remember as historically sexy,” Erika Kinetz wrote in 2005 in The New York Times, reviewing “Giordano Moves,” a tribute presented at the 14th annual Jazz Dance World Congress in Chicago.
Gus Giordano, 84, Innovator of Modern Jazz Dance, Is Dead – New York Times

Pawn remembers Gus as friendly and open, and very respectful. He had already won his Emmy award by the time I met him, but was gracious and down to earth. His company loved him, and it showed in the enthusiasm of their performances. I always looked forward to their arrival at the theater, and learned a lot about lighting design working on those shows.

Old Tech On A New Frontier

The Stirling Cycle engine is a nearly two hundred year old invention which has had new currency of late. Pawn experimented with these efficient little puppies as long ago as 1973 (Yikes!) and even presented a paper at an international conference on the subject of solar powered Stirling engines over 30 years ago, as a young pup. Now NASA is looking at using a Stirling Cooler, which is simply a Stirling engine run in reverse as a heat pump, to cool Venus landers.

The surface of Venus broils at a temperature of about 450 °C – hot enough to melt lead. Several probes in the Soviet Venera and Vega series, as well as a NASA Pioneer Venus probe, landed on Venus and returned data from the surface in the 1970s and early 1980s. But they all expired in less than 2 hours because of the tremendous heat.

Now, two NASA researchers have designed a refrigeration system that might be able to keep a robotic rover going for as long as 50 Earth days. The work was carried out by Geoffrey Landis and Kenneth Mellott of NASA’s Glenn Research Center in Cleveland, Ohio, US.

The main concern is keeping the electronics cool. The NASA pair plan to do this by packing the electronics in a ceramic-based insulator and placing it inside a metal sphere about the size of a grapefruit.

Heat would then be pumped out of the sphere using a Stirling cooler, which works by compressing and then expanding a gas with a piston. When the gas expands, it cools down, absorbing heat from the electronics chamber. Then, as the gas is compressed and its temperature rises, the heat is allowed to dissipate in the atmosphere via a radiator.

Stirling coolers were invented in 1816 by Reverend Robert Stirling, a Scottish clergyman, but were largely ignored until the mid 20th century, when their impressive energy efficiency became better known.

Antique fridge could keep Venus rover cool – space – 12 November 2007 – New Scientist Space

Thanks to the eagle-eyed folks over at Slashdot /. for pointing out this story.

The Church of Latter-Day States

handcuffed.jpg

Remind me again why we granted statehood to Alaska and Hawaii (just kidding)…

US senators today made a bipartisan call for the universal implementation of filtering and monitoring technologies on the Internet in order to protect children at the end of a Senate hearing for which civil liberties groups were not invited.Commerce Committee Chairman Daniel K. Inouye (D-Hawaii) and Senate Commerce, Science, and Transportation Committee Vice Chairman Ted Stevens (R-Alaska) both argued that Internet was a dangerous place where parents alone will not be able to protect their children.
US Senators call for universal Internet filtering | Press Esc

USPTO Sanity

seeds.png

Every once in a while the US Patent and Trademark Office gets one right. They have just done so by tossing out four patents previously awarded to Monsanto:

Monsanto has filed dozens of patent infringement lawsuits asserting the four challenged patents against American farmers, many of whom are unable to hire adequate representation to defend themselves in court. The crime these farmers are accused of is nothing more than saving seed from one year’s crop to replant the following year, something farmers have done since the beginning of time.
PUBPAT > Monsanto Patents Asserted Against American Farmers Rejected By Patent Office

And now for this brief announcement

advertisement-message.jpg

/. has brought this Ars Technica article (covering a story first broken by Information Week) about Micro$oft receiving a patent for a technology to data-mine your hard disk for advertisers. Makes me glad I don’t use Micro$oft products.

Microsoft has filed another patent, this one for an “advertising framework” that uses “context data” from your hard drive to show you advertisements and “apportion and credit advertising revenue” to ad suppliers in real time. Yes, Redmond wants to own the patent on the mother of all adware…
“Applications, tools, or utilities may use an application program interface to report context data tags such as key words or other information that may be used to target advertisements,” says the filing. “The advertising framework may host several components for receiving and processing the context data, refining the data, requesting advertisements from an advertising supplier, for receiving and forwarding advertisements to a display client for presentation, and for providing data back to the advertising supplier.”
The adware framework would leave almost no data untouched in its quest to sell you stuff. It would inspect “user document files, user e-mail files, user music files, downloaded podcasts, computer settings, computer status messages (e.g., a low memory status or low printer ink),” and more. How could we have been so blind as to not see the marketing value in computer status messages?
Microsoft patents the mother of all adware systems