Gestural Voting System, A Public Interface

Introduction:
Over the last two weeks, I thought a lot about an appropriate public interface that would allow people to engage in a meaningful dialog in a common space. After toying with animated web pages, RFID tag systems, and a postsecret-like public installation, I realized that I needed to simplify things and design a more simple system for people to suggest questions, and playfully answer them with their bodies.

Consider the following scenario: we are all going to attend a conference on sustainability. We want to include a surveying system would allow for everyone to contribute to a sustainable questions base, answer other peoples questions, and afterwards be able see the overall aggregate of the public responses.

Background:
This idea was inspired by a book called "The Wisdom of the Crowds" by James Surowiecki. In the book, New Yorker columnist James Surowiecki explores a deceptively simple idea: "that has profound implications: large groups of people are smarter than an elite few, no matter how brilliant—better at solving problems, fostering innovation, coming to wise decisions, even predicting the future." (from the random house description)

A second inspiration for this project is Mickey McManus of MAYA design firm a creative design consultancy and brainstorming firm Pittsburgh on the phone last month, and was very grateful for his advice and guidance regarding realistic systems for creative group innovation. He makes note of their theory of Average Innovation, which involves taking an aggregate average of the overall response of an innovation group.

At a conference gathering where people go to meet each other and solve domain specific problems, many important questions are not asked for political or social reasons. But if a system were available that allowed for the collective cataloging of peoples thoughts opinions to be presented anonymously and democratically it would satisfy many of the criteria of a successful brainstorming session.

The System:
There would be 3 stages in the system, Registration (collecting questions for the crowd), Public Interface (simple gesture based voting by people attending the conference), and a Follow-Up System (results, both public and private).

Step 1. Collecting the Questions
During registration each attendee enters one question with a number of options for viewers to choose from in the a public lobby. This gives people the freedom to anonymously ask a question of the public that they care about, contributing to the content of the installation, and eventually getting an answer back.

Selectricity has an open API under development so that "programmers and hackers may want to use Selectricity from within their own site or to embed Selectricity functional into their own system. Selectricity makes that simple with a featureful API and the ability to 'embed' elections."

I love their design and system, and have hacked it a little bit here to show how simple it would be for people to contribute questions:




Step 2: Public Installation
The video below this paragraph shows my working prototype for how the interface for gestural voting would work in the space. Ideally, the projection would be much larger, and accommodate more than one person, but I was able to hack up this version in Processing over the course of a few days. The participants would know that their question is embedded, and hopefully each person would answer a few questions in the space, by raising their right or left hand.

Artists like Camille Utterback, Scott Snibbe, Zack Lieberman, Golan Levin, and David Rokeby all use the body as the primary interface to digital content. Scott Snibbe notes in a 2006 inteview with Roberto Simanowski of Brown that, "Once I began showing in museums, I realized that the primary way anyone engages with art is by approaching it with their whole body. I then aspired to create artwork that is engaged entirely this way – by walking and moving through an “aware” space that responds to you." His notion is that first we engage the body, and then our minds. For me, gestural voting is reminiscent of classroom polls from my childhood where the teacher would ask us to raise our right hand if we understood, and our left if we didn't.

click to open (rough) demo video, 1.08 min

Step 3. Viewing the Results
Again, using an internal version of Selectricity's results matrix - there would be two levels of results feedback. First, each attendee would receive an e-mail with a link to the results during the conference. This would be a private e-mail. Second, anonymous results with all the questions published would be available to everyone - and would act as an overall survey of issues pertinent to the occasion, democratically surveying everyone in a playful way.



Conclusion:
The Gestural Voting System satisfies a number of challenges I have been struggling with when trying to design a interactive social installation. It allows for creative generation of content by the people who will use the system, simple physical interaction with the public installation, and and a comprehensive results interface. The balance between public and private is satisfied by each stage - I envision the system adding a layer of depth to a conference or social gathering that would compliment the event and even serve as a platform for further engagement around key issues for the group.

Addendum: Features and Suggestions Notes
Suggestions for Improvement by Judith & Others:

A) More degrees of choice, utilizing the spectrum of choices from Selectricity.
B) Having the Background respond to each question answered - to come alive and reward engagement.
C) Adding an affective surveying element to the voting system that would say things like "Are you enjoying this survey?". This could help normalize the results - if people are self conscious or not.
D) Add Icons to help with hand raising for first time users.
E) Think about the appropriate installation space, if more than one user could use it at the same time than a line would not form behind the piece.
F) Would you want to show an aggregate after people answer the questions as a way of providing feedback in the public installation?

On Mon, May 12, 2008 at 9:17 PM, alyssa wright wrote:
Hi Seth,
The project looks really good, though of course I'm partial to any selectricity development. If you're interested in working further with selectricity you can always talk to Mako (if you haven't already) or myself. We're doing work on ways of visualizing the votes, and your physicality makes perfect sense. One thing of note, that you might have realized already. . . but the algorithms behind selectricity are best for questions where there is preference. Yes/no questions aren't the best use of selectricity, though it can definitely work in that capacity. So a question like "do you recycle?" might have more variagated results if it was a question like: "what do you think is the most important items to recycle?'" and then options like 1) paper 2) plastic 3) food 4) glass 5) nothing (or something like that.)

Hope that's helpful. And hope you are well,
Alyssa.

Hi Alyssa,

Thanks so much for the feedback on this - I'd like to add a feature to
the system which would allow degrees of feedback or one of 5 movements
through subtle uses of the hands. I agree with that suggestion.

Also - yes I'd love to be a beta tester for the API if Mako is open to
that - perhaps you could forward my proposal to him? I'm so impressed
with the work you guys did on that...

Supertraces - Discussion Questions, Public & Private Data

Discussion Questions For Class Tuesday:

1) Steve Mann's inventions empower him as an individual - giving him a means to question authority within it's own constructs. (laws, technology, personnel hierarchies, power dynamics). Mann is pointing out a strong relationship between power and technology. Like the Critical Art Ensemble and other technology artists - he protests the use of technology for surveillance by exaggerating it's current use in society. What means do we have to resist and protect ourselves without being radical activists?

2) The notion of a diagrammatic view of panoptic surveillance, argues that consumers are rewarded by software agents that present familiar images and commodities, and punished by having to work harder outside the system to find obscure or unfamiliar items. Just in Time Systems operate on the principle of suggestion by popular aggregation (others who purchased this) - search terms, and previous purchase history. Does such a system homogenize our thought activities and streamline popular thought or do they provide a means of seeing how others approach the same topic. What would a "decentralized" database or search engine look like? Would it be able to make suggestions without losing context?

3) Given the Murky Nature of Public and Private data (seeker and subject, norms, power roles, expectations of both parties) and the complexity of "user agreements" when we enroll in services, perhaps it would be advantageous to have a GPL (GNU General Public License) for commerce and surveillance. The advantage of the GPL is that most software developers are familiar with it, and can readily agree to it's terms without having to review a long agreement. The expectation that any user will read a term of agreement is unrealistic, so an open-surveillance agreement which protects both seekers and subjects from each other and from larger powers like the federal government, may also limit the capabilities of surveillance technology, or jobs from monitoring our activity. What would such an agreement say, and how could it adapt to multiple contexts? Do people think this is a good idea?

4) In "Born to Buy" a form of modern child labor is described where kids go out and collect information for large companies on consumer preferences of their peers. Generally, the ethics of these practices involve a parental consent form which indemnifies a company from being held liable or having to pay a child whose idea becomes the next profitable product. Children enjoy being listened to and having an opportunity to earn a small fee. Is such a practice morally acceptable? How might labor laws allow children to participate in creative contribution to a market and be sufficiently reimbursed for their efforts if the ideas are successful? What laws are in place to protect the privacy of children who cannot know the implications of their actions?

Social Textterflys - A Shared Screen Design

I've seen quite a few calls for proposals with Urban Screens - or social screen designs. So the other day I had an idea for a shared screen where static web content from each persons profile seems to "come to life". In essence the idea is to transform what we perceive as static social content into dynamic and contextual content in a social space.

Using technology in social networking - or semi-public spaces where people are trying to get to know each other - is a delicate balance of public/private - gimmicky interest matching - and often interrupts the natural flow of conversation...

I would like to make a playful, large touchscreen for a wall where people go up and scan their RFID nametags - and a weblike profile comes up (which they prepare when registering for the event). THEN it slowly comes to life, your interests turn into birds and fish which
fly and swim over to other people's animated words. The whole profile disintegrates and all the elements either drift off or join an ecology of abstract connections between people. I imagine people could walk up and playfully interact with the birds, nudging them to fly or find other connections.

Here is a 3 minute video outline of the idea. I thought I would try this format as an experiment in place of doing image layout.


Writing the code is outlined above. My Proposal is to have a prototype of the concept for the final in this class. Something that brings up image profiles, and then proceeds to break them into image maps that have behaviors.

A System for Facilitating Interaction in Communication Spaces

Judith notes an important distinction in the "Everyday Interactions" paper regarding technology as an "agent or medum". The metaphors and expectations attached to these roles fundamentally change the way we interact with the system. If the computer acts as a facilitator between people it has a much higher chance of being adopted as a real "everyday" mode of interaction. The system as autonomous is magical, but it's novelty quickly dissipates when it is understood, and further interactions simply reveal the algorithms behind it's behavior.

How can we integrate computational systems to more meaningful, human systems in an inherently more personal (but not intrusive) way? What are these channels and systems of meaning? A few that come to mind are writing, touch, gesture, speaking, the making and exchange of objects, and social traditions of interaction such as eating, theatre, concerts, parties. I want to start experimenting in the domain of large social gatherings, without asking people to alter their activities in any way during the meetings, but augmenting the experience and providing a simple architecture for preparation and follow-up after the social gathering.

Guidelines based on the readings I considered in designing the system are:
A) Balancing the public and private aspects of the interaction. Perhaps one approach to this is to separate the actual data exchanged in the interaction from visual response of the system - by abstracting the data and translating it's form.
B) Not interrupting the natural social flow, or introducing new ways of interacting. Use our existing social framework as an informant for behavioral interaction.
C) Augmenting and rewarding behavior that is social - results in an exchange between two people. Handshakes, Proximity, Time, and Contact information exchanges are all forms which indicate the investment and intimacy of the exchange.
D) Focusing on reviewing data as a private activity - your profile (pre-meeting) and social visualization (post-meeting) as records for you to use as follow up tools. Focus on strengthening the "Broader Social Sphere" - as noted in the cell phone reading.

------------

ourSpace, myRing, myCard, mySocialMap - simple, seamless tools for tracking and augmenting interaction.

myRing is a small battery powered circuit with a Bluetooth radio and an RFID proximity sensor. When you come to the meeting you pickup a ring at the door by scanning one of your myCards. myRings are activated by handshakes (proximity) - and also report position data through triangulation by bluetooth receivers. I'm not the first person to think of this form for blue-tooth device, see: http://www.bluepoly.com




myCards are aprox 100 Disposable business cards (contact cards) with simple RFID tags imprinted in the card. Printed from your "web profile" and available at checkin, they serve as a tangible means of exchanging more detailed an intimate information.

These exchanges are measures of proximity and interest - and coupled with proximity sensing - they could act as triggers for pulses of animation and liveliness within a space.

ourSpace is a series of responsive projections in the periphery of the social meeting. Each time a business card is exchanged, or people shake hands, the walls come alive - Plants growing, fish swimming, an entire ecosystem is activated and responds to the activity and history of the space. The visualization is abstract, and immediate - a positive and fun reinforcement of the physical activity in the space, without revealing the identity of the viewer.

Below is a simple view of 3 possible types of projection in a space. (I would use one theme, but I tried plants, snowflakes, fish as examples)



mySocialWeb is a visual record of the night. It includes nodes for each person I shook hands with and more detailed information for people I exchanged business cards with. Business cards can be scanned at home for digital replicas, and links to public web information from the person whom you just met. mySocialWeb is a private web service to help facilitate social relations by allowing for quick follow-up and longterm records of social encounters at a conference, festival, party, art opening, dating function, or business meeting. It is intended for an Urban setting where exchanges are often not repeated and reinforced unless contact information is exchanged at the time of meeting. (in a small town setting such an interface would be intrusive and unnecessary)

Portraiture Follow Up, Combining Ideas

I continued to experiment with the software during the week - with friends. Kate and I were playing and discovered that if you use a background image with text - the juxtaposition can be quite interesting.

Here is a progression of portraits - The earlier idea of combining something someone has written into the portrait was the theme for the last few takes.




I asked Kate to experiment with the software - she picked up a coupon from the table and played with that format.




Then on the subway over to Cat's house I thought about using personal writings... and moving light around her head in the background to try and get more interesting difference matching. I asked her to pick images that she felt reflected her interests & desires..



Kathryn took a few pages from her diary here and watched how the software rendered her - I like these the best of the portraits I have taken so far - There are hidden gems in the text and she was open to the idea of seeing herself in the writing.

Information Portraits: Two Ideas

Idea 1: Instead of Taking a portrait of what is there - do the opposite - only look at those elements that are displaced. The idea was to write some software that would measure this as at a granular level and then blend together the results. I wrote this code in processing and used it to make some self portraits during my experimentation.


My second idea was to have text portraits. This is a mock up of the idea (in short) - I would like to do this with 10 layers of transparency - so that you only see the text at first and then from 1 angle everything comes together magically. So you could read what this person has to say about themselves, and then their own words become a layered portrait.

Next week after the demo week is over, I plan to try some of this by printing onto transparency, but for now I hope this will do.

The Fabrication of Idenity

The meaning of a portrait is entirely dependent on the social, cultural, and historical context of its creation (the artist), commission (sitter), and viewing (audience). If one were to glean anything from Brilliant's writings on portraiture perhaps it would be that identity is a fabrication of A) the "artist's imagination", B) "desire and role of the subject", and C) the social context of the portraits presentation to society.

Portraiture in this way is a framework for defining the relationship between the self & the world. It is about capturing an aspect of the nature of identity, and re-presenting for a social purpose. We present and represent the self as Art, Biographical Documentation, Historical Information, Personal Memorabilia, Political Iconography, Memorial Imagery, the list goes on.
In light of the multitude of roles it can play, from the empty images of Marolyn Monroe to the iconic & stoic portrait of Weber, how will the tools and context of the information age transform portraiture? Will portraits still try to capture a "timeless" self, unique and yet comfortably portrayed in the generic type roles of their day? Does interactive portraiture allow for a more "authentic" view of identity?



Oscar Wilde said that Portraiture is really a portrait of the artist (p. 82). Jon Gerrard attempts in his works to overcome the subjectivity of representation by presenting super-high resolution images of his subjects in three dimensional views with Matrix like 360 degree rotation. The viewer can change the angle of viewing by rotating the video image along an arc in the gallery.

The feeling I got in the gallery was of a sort of hyper reality, where the iconic role of the subject is lost and the sense of a likeness dissolves into a clinical view where one sees many more details than you might in your everyday perception of the same person. We see people in different lights in our everyday lives and as locomotive creatures have a holistic representation of their form, but we don't often flatten this perception into a massive quantity of data and present it in such a surreal way.



Gerrard and Bill Viola both use portraiture in this way to evoke how the role of technology can amplify our experience of portraits by slowing down time and increasing detail - somehow our iconic representations of each other are lost and we move beyond our idea of identity. Shearer West said that portraiture is about body and soul - these portraits seem to leave the soul behind the facade of the exquisite copse.



I have also been living and working with a portrait series by Orit Zuckerman called Influence. I'm sure everyone at the Media Lab is familiar with it - to me it is the most authentic representation of people in a social context that I have ever seen. Influence is an interactive artwork visualizing how collective behavior emerges from decentralized interaction in a small social network. Individual people are affected by the behavior of people around them, and as a result, they influence the people around them as well. As I have lived with it I have come to appreciate how accurately and sensitively Orit and Sajid approached their subjects - and how considerate they were about how the subjects were presented. A remote control interface allows a viewer to explore the personalities and reactions of each viewer.

So in each of these examples I would say that yes, they function as portraits - but not in the same gnomic way of the past - instead they posit new technological and social contexts for viewing a person and place the act of portraiture into the role of observing itself. (as Escher did but by placing himself in the eye of the viewer).

Time, Memory, Representation

Click to Enlarge, 1024 x 4400

Sunday from 9:30 to 3:30 I tried to pay particular attention to the time, documenting it in 3 ways. It's been a year and a half since I have taken pictures. This is in part because I am curious how people are affected by the act of photo taking and how much this dominates their memory of things (does our memory become mechanized). I think this is critical find more flexible ways to show how we perceive representation of time from a more holistic standpoint.

From top to Bottom in the Collage you can see the Progression from (1)Memory to (2)Documentation to (3)Visualization of Time.

1) Memory, Pre-Photo Review - What do we remember before we allow our photo taking to structure our memory? I jotted down everything I could remember on the bus ride just after the 6 hour period elapsed.
2) Documentation, Photo Review Process - (see middle section above) - How different is the feeling of the story without the spacial and undocumented aspects of memory, feelings, ideas, mental models.
3) Visualization, Post Memory Collage. I tried to imagine if you could reorganize the layout of a photo show to emphasize the subjective emphasis or as roland barths would say - the "puntum" in the image sequences.

Online & Offline Conversation

Online conversations are more direct and succinct. Social convention requires less formality and often encourages a playful invention of new syntax - a short hand convention that tries to make people feel like the space is slightly more intimate. Often it is understood that interruptions and delays in response are normal - but social norms encourage people to explain the delay with a short phrase that acknowledges the interruption.

I thought it was interesting how he noted that most conversations are not about the information being obtained, but about reinforcing relationships. I would argue that online, people seek information more than companionship - or they are interested in building relationships that will later be person to person conversations. In forums the speaker addresses everyone - but in life we make eye contact with one individual at a time and there is a sphere of influence in our conversational realm.

I tried to create a dynamic animation of such a conversation where two people are visiting a three person home to inquire about possibly living there as well. In the text landscape you see spheres of influence, the amount spoken, and who the speaker is. Online, the relationships develop in a more web-like fashion, the spheres of influence are broader and more democratic.

Here is the animaiton of the conversation (I can't embed swfs here):
http://web.media.mit.edu/~hunters/personal/conversationMap.swf



And here is a conversation from an online space, click to open.

Response 2: Interactive CoMedia Shared Histories

Please Click to ENLARGE!

CoMedia Shared Histories - An Interactive Chat for SubTextual Comments & Mutual Viewing

In considering a context for a 2-D chat environment Kate & I realized that we would only really want to participate in a space if it was very context specific, a defined topic with layers of rich cultural interpretation and previous discussion. Kate mentioned that John Maeda once said to her that the problem with the web is that it is Lonely. We can't see that other people are there. I agree.

CoMedia Shared Histories, please click to see the full design, presents an interface for doing the following:

MACRO:
1) Choose a realm for context of interaction (SEE TOP BAR):
a) Share, Collaborate, View the same information (documents & Files & Desktop)
b) Add SubText to Services connected to Media (Flickr, NetFlix, GenArt) CoFlix, CoGen
c) Poll or Aggregate opinions on in a Geographical Space on Specific topics.

2) Filter the History of Information by:
a) Related community - friends who share similar or disimilar categorical interests, or the case of the CoFlix - genres, or values.
b) Filter by VALUE PROFILE, things that you prioritize in your search for new or existing material - authenticity, dialog, character dev, natural lighting....
c) Filter By Date, Rating. See Drag bars on the top and bottom of the picture.

Filtering provides a way of turning on and off different layers of information, and adding comments to those context specific layers. People interact in a Geographical Space that reflects their History, Preferences, and links them with similar people in a large community.

MICRO: Viewing Specific Content within a Distribution Matrix:

In this case I have choosen to illustrate CoFlix - a system for mutual viewing of a NetFlix movie at the same time. Within this application viewers can focus on the following:

A) Specific Points in the Timeline of the Film, and types of comment layers that might show up within that Chapter or Scene.
B) Interpretations and Directors Commentary - comments that have been signified as valuable and integral to the production.
C) Meta Relationships and Links to similar scenes - and peoples thoughts around those.

As people interact with the system it records their preferences and updates their Profile History. This provides a framework for determining where their comments might show up later for other people looking at a film a second time and leaving their own notes and comments.

The depth of such a system allows for Realtime Interaction and Chat - but also builds a History of interaction with the Medium that is context aware. This would solve the problem of aimless chat and focus the discussion on the medium at hand, utilizing the most immersive medium in popular culture, film.

Response 1: Social Metaphors

Knowledge is Discussion (forums) is Community is Information is Knowledge

Beyond Being There stipulates that we need to focus on the needs behind our forms of communication over trying to recreate face to face communication: “it is not so much distance that will be abolished, but rather our current concept of being there.” The Archival, anonymous, and semi-synchronous benefits of a system like a forum or an electronic billboard do go beyond being there, but must be rooted in a common goal that either requires tools to manage it’s complexity, or tools to manage the distance between us. Rather than phrase the question in terms of “being there” or going beyond reality - I would sidestep the question and phrase the problem in terms of “tools that seamlessly integrate and enhance our current forms of communication”. The effort to create forums that united distributed and specialized communities, and push to create mobile devices that can exchange logistic information and integrate our digital a physical lives are focused on this type of cooperative relationship.

Forums like the New User Interface Group (NUI), a social network for the development of multi-touch tables that I am a part of, use the metaphor of the “forum” an architectural space of Gecko-Roman origin, as a means of information exchange and discussion. They rely heavily on the advantage of searchable archives and shared problem solve for the DIY communities of hackers around the world. I might lack social compatibility or even language compatibility with many of the people in the forum, but we share pictures and links as a way of enabling community research. The interesting metaphor that seems most present in the forum is a pseudonym, this acts in part to allow “newbies” to post entries in the forum without being shy, and administrators to appear informal - we are all in this together with a common goal and initiative. Not only a first name basis, but nicknames and syntax reflect the democratization of information that has become the tacit goal of many online groups - to provide a “bottom up” distribution of information that allows anyone to engage. This requires the initiative and vision of a few committed organizers and low level voluntary commitment of a larger community.

The forum approach is limiting in the sense that “topics” of conversation often start “threads” that are linear and users will complain that a topic has already been discussed. Issues are not relevant once they have been solved. Another difficulty is that the forums don’t provide a means for determining if an issue is solved, or summarizing the outcome of the discussion. People have to dig through the entire archive of correspondence of all the members to understand context - even when a search engine brings up results of a particular thread the things that come up don’t necessarily correspond with the heading of the first post. Similar dynamics occur in a classroom when one or two people initiate the topic of conversation and don’t allow for a lot of deviation from the threads.

---

Icons as Visual Metaphors: a database of iconic images?

The concept of vision as “the primary medium of thought” - and that without the nervous systems activity the mind does not function well, is reinforced by the efforts we have to present information in a visually dense but informative manner - using less symbolic constructs than text. This led me in a new direction of thought regarding visual representation:

Has anyone ever created a visual database of icons in vector or scalable format? A collection of cultural images that by popular vote that are simple line drawings in and shape forms immediately recognizable as our modern system of hieroglyphics?

I'm looking for images with a uniform look and feel that could be substituted for words when people speak. What if I translated your words into images realtime? Would that process make you think of the content of your words in different terms? If such a database existed and could be accessed like a wiki - wikicon would be the title of the project.

If an icon is an essential visual metaphor - the treatment of images would rely primarily on peoples semantic and historical notion of concepts: see Building a Visual Database for Example-based Graphics Generation (http://portal.acm.org/citation.cfm?id=857735)

If shapes are concepts that icons are refined from a collective consciousness (gestalt approach) . I want to utilize an icon browser in some way that harnesses the intelligence of our perceptive minds... we hardly ever see out of context - p44 (Visual Information) - so why not transform or interpret the context realtime?

What underlying metaphors would such a system reveal about the manner in which we speak and our visual representations of those concepts? If concepts are structured in metaphorical terms, ie Love is Acceptance or Love is a Journey - It would be interesting if someone typed Love and saw images of those Metaphorical associations: IE: A hug or a person traveling down a long road come up when someone types love. Surprising and thought provoking results? We sometimes understand concepts in new ways through images - in a more cohesive way than we do when replacing them with more words.