Showing posts with label Processing. Show all posts
Showing posts with label Processing. Show all posts

Tuesday, 6 November 2012

What is @twitr_janus - A summary overview, November 2012 for MCN attendees

This post is an updated summary of what @twitr_janus is, and what it demonstrates

This is mainly aimed at attendees at the Museums Computer Network conference, Seattle, 2012, especially the Slack Day hacking workshop, where this cutey will be being demonstrated live. (if I can put it back together again)

Firstly, here is @twitr_janus in cabin baggage ready to be checked in as hand luggage. I am not sure it will be allowed on board the aircraft...
@twitr_janus skull

What is @twitr_janus?

DSCN2094

@twitr_janus is a live physical avatar - a puppet that can be controlled remotely over the Internet using the data sources of common web services like Google Drive and Twitter and direct web-enabled services such as Skype. It is a (very) crude alpha prototype that demonstrates several simple ways to remotely interact physically using web data services or piggybacking the connectivity of audio/visual web services.

Its real point though, is to demonstrate how real-time sensory interaction can be achieved by hacking together feeds, APIs or just features of common freebie web services. It can see you, hear you, talk to you and move its eyes to watch you. That's not creepy at all.



How does this work?

@Twitr_janus electronics To control @twitr_janus, data is sent to it over the Internet using free web tools that have interactive elements. These include sending a tweet in a specific Twitter account and creating custom forms that piggyback the form processing script of a Google spreadsheet, but many other web services could be used.

At the other end of the web, the structured data is retrieved so it can be manipulated. This is done using the open source Processing language running on a connected computer. Processing can parse (strip out selective data) from open RSS feeds and/or APIs.

The data is then converted into physical action. An open source text-to-speech library for Processing is used to generate speech from text data. It can speak either tweets from its Twitter account, or from the contents of a field in a Google spreadsheet.
@twitr_janus uses the open source hardware platform Arduino to convert other data to physical motion. Eye control is achieved by an Arduino sketch loaded on the board, which reacts to data parsed from feeds from a Google spreadsheet. This signal data controls the position of two servos that direct the focus of its stare.

Its jaw is lip-synched to speech output by converting the analogue audio signal levels into a control signal for a car-door lock actuator motor. An audio amplifer is used to create asignal large enough to detect, and the Arduino sketch triggers a relay power circuit for the actuator based on signal peaks. It also has LEDs to indicate the last source of its control data.

Finally it can see and hear via a webcam. It does this using web video software Skype. This can be remotely activated using the correct settings in Skype

WTF?

Burnt out servoThe important point is simply that almost any modern web service will use, or often be built upon, a data publishing model. Because these web services have built-in data publishing features, such as APIs (application programming interface) or feeds such as RSS, they convert the human input tools into predictable data output that can be manipulated. None of this requires you to set up any infrastructure.


This principle can be used to hack new things together. It's all very enjoyably Frankenstein. @twitr_janus is just a specimen beast, created to illustrate this with a physical example. It simply demonstrates some possibilities beyond obvious web front end stuff, of hacking data from current open source or freely available web services, to create physical interaction.

It's crude, often breaks and has cost the lives of two Arduino boards, at least five servos and one vintage glue gun (RIP), but it does show how web data doesn't have to be just on-screen.

Other things that could be quickly prototyped using this method can easily be imagined such as:
  • Tweetometer for marketing campaigns. Use Twitter data about your campaign tags to make a physical thermometer go up and down to indicate success. (e.g. Trend ranking, number of retweets, etc.)
  • Animatronic avatar interaction to query collections or events databases. If data can be extracted, it can be spoken
  • Physical puppet automatically announces new events as they appear in the api of your events database
  • etc. etc.




Free code and stuff...

Twitr Janus tweet retrieving script in Processing(with apologies to my professional developer colleagues for the state of this hackery!)
The basic (and very rough) code is available for forking/hacking/recycling as you see fit. Note, this will be only the last working version:
Processing control sketch (version 15)
(polls web feeds from Google and Twitter to extract data)
Github code download
PDF version
Arduino sketch (used to manipulate data and/or analogue signals into physical actions)
Github code download
PDF version

There are detailed posts about the specific physical build of @twitr_janus in the archive if this blog. Physical build notes crop up in back posts here on makingweirdstuff (interspersed with other nonsense) 
E.g. Posts about building eyeballs

Friday, 7 September 2012

MakeyMakey eyeball interface controlled by cardboard and a coin

Well, I have to say that the MakeyMakey is pretty f. awesome and the whole idea of crowdsourcing a product via Kickstarter is really quite superb.
And so, let's build something...

Here is a nice physical cardboard eyeball (about 8cm x 3cm)
MakeyMakey DIY eyeball UI
The eye was knocked up in about 40 minutes from card, paper, pencil, Promarker felt tip pens, glue and a two pence coin...
MakeyMakey DIY eyeball UI MakeyMakey DIY eyeball UI














Here is that very coin - glued aback a cardboard ring, which is the iris.
MakeyMakey DIY eyeball UI MakeyMakey DIY eyeball UI














On the left is the completed layered eyeball. At the bottom, you can see a crocodile clip...
On the right is the cardboard eyeball, connected to the MakeyMakey, connected to the PC which is running the virtual Twitr_janus_eyeballs Processing sketch running on openprocessing.org
MakeyMakey DIY eyeball UI MakeyMakey DIY eyeball UI














Below on the left is the easy-peasy MakeyMakey crocodile clip connectors (I love crocodile clips)
Right - twitr_janus eyeballs
MakeyMakey DIY eyeball UI MakeyMakey DIY eyeball UI














Or you can just watch a fairly short demo video...
This took about 90 minutes to set up from nothing to working. SWEET!!

Friday, 17 August 2012

Pimping a Google spreadsheet form, to create a bespoke control form with sliders

Link here to download complete Processing sketch on Github

This expands on the previous post, which explained how to use the RSS feed from a published spreadsheet to extract remote control data from that feed and use it in a Processing sketch.

Here this idea is extended to use a Google form, instead of typing into the sheet. This has the advantage that all the values to be sent can be prepared at once, then the form submitted.  It is still a bit clunky, but it does imporve it slightly. A mthod of controlling the inputs to submit data between fixed upper and lower values for both horizontal and vertical position variables is described below.

The Google spreadsheet showing form data and control data based on the last form entry

Twitr_janus Google spreadsheet


The values in the fields F2, G2 and H2 are the data that will be used remotely by the sketch.  The data in these fields are being constructed from the control character ¬ in E2, and the last instance of a form return from columns B (representing up/down eyeball data), C (representing left/right eyeball data) and D (speech text) for the Twitr_janus puppet.

The Google expression in cell F2 to return the last NUMERIC value of column B is...

=trim(E2&(FILTER( B2:B ; ROW(B2:B)=MAX(FILTER( ROW(B2:B) ; ISNUMBER(B2:B))))))

The Google expression in cell G2 to return the last NUMERIC value of column C is
=trim(E2&FILTER( C2:C ; ROW(C2:C)=MAX(FILTER( ROW(C2:C) ; ISNUMBER(C2:C)))))

and the Google expression in cell H2 that returns the last TEXT value of column D is
=trim(E2&FILTER( D2:D ; ROW(D2:D)=MAX(FILTER( ROW(D2:D) ; ISTEXT(D2:D)))))


The puppet is not yet built, so a surrogate virtual face is being used to react to the data. It is rather basic...

Twitr_janus' virtual face
The standard form for the spreadsheet was set up as usual and has text boxes. This is  slightly better than direct typing into the sheet, in that the 3 values can be sent in one go. However, it requires the user to have to know the values needed to make the eyeballs move and allows any value to be submitted, which will cause problems.

To get round this, the source code of the form needed to be adjusted. It looked like this:

Source code for standard Google form
















The post method used here will only work if you are logged in as an editor of the sheet, but if you are logged in the form code can be used on any webpage. This means you can copy it and hack it, then embed on your page.

Source code for the pimped Google form



Original dull ole' Google form
Twitr_janus
Beautiful pimped form with
range limited sliders - nice!
The main thing to note here was the adjustments to the input controls for the two eyeball data fields.
The original input has the attribute type="text". This is shown on the left astext boxes.

The pimped version on the right (from an iPad screen) uses the attribute type ="range".

Range is an HTML5 feature that displays a slider.

It also allows for min and max attributes to be set, which limit the data entry to between those two values. Just what was needed!

ta-da!



Wednesday, 15 August 2012

Remote-control eyeballs: Using remote data from a published Google spreadsheet to position eyes with Processing

Twitr_janus virtual eyes

Above is shown a simple keyboard controlled Processing sketch in action. The arrow keys are used to position the way the eyes look. Each arrow key increases the value of a position variable for the irises and separately for the pupils of the eyes.

To remote control Twitr_janus' eyeballs, this was extended to send position data in 2 directions based on the values in a published google spreadsheet. The RSS feed is read and parsed by Processing to extract the data, which is used to compare the current horizontal and vertical positions with the target positions received. Processing runs a detection function that detects any difference, and reduces the difference by incrementally adjusting the current position and redrawing the eyes, until it can't detect a difference. A classic cybernetic negative feedback system!
Twitr_janus eyes controlled by Google spreadsheet data Twitr_janus eyes controlled by Google spreadsheet data













Left published Google spreadsheet. Right, same with eyes.


Twitr_janus virtual eyes Twitr_janus virtual eyes













Initially, so attempts at this went a bit haywire, due to over thinking the script to work out boundaries to stop the irises  and pupils doing strange things (left) or leaving the edge of the eyes (right).

This java applet demonstrates the direct keyboard sketch in action. This is freely available to download and copy at http://openprocessing.org/sketch/67233

It requires Java, so won't work on some devices, but if you have a Java plug-in, you can make the eyeballs move with the keyboard arrow keys...

Thursday, 9 August 2012

Twitr_janus is now speaking data sent from published google spreadsheet (via RSS feed to Processing)

Twitr_janus' brain can detect more data

I have finally managed to work out how to get Processing to respond to data live from a Google spreadsheet. The effect of this is that I can send separate multiple packets of data to Twitr_janus from a single source, to control multiple functions.

This is a significant development, as data can now be sent for different functions such as text to be spoken or positioning information for eyes, or other body movements.

I have made a successful test of sending 3 packets at once to the latest version of the Processing sketch (Twitr_janus' brain), which it can decode. These were:
  • A number that could be sent by Processing to the Arduino, to tell it how to position a control servo for an up/down eyeball movement 
  • A similar number that could be sent by Processing to the Arduino, to tell it how to position a control servo for an left/right eyeball movement 
  • A text string that was successfully received and converted to speech. 
Here is Twitr_janus's current voice, speaking data sent from a Google spreadsheet


download: mp3
"This is Twitr_janus speaking thoughts from a google spreadsheet"


download: mp3
This method of getting data from my master means I can speak much longer lines, and not worry about stop characters


download: mp3
"Oh yes! and the sending of other data from the same google spreadsheet works. This means my eyeballs could be made to move."

How Processing gets the data from a remote Google spreadsheet via RSS


Setting up the spreadsheet

A Google spreadsheet was set up to hold the test data. This had three data fields for ENTERING data:

  • eyeballUpDown (the field name says it all really) - data in A2 in the spreadsheet
  • eyeballLeftRight - data in B2 in the spreadsheet
  • text (this is the piece of text used to make Twitr_janus talk -  data in  C2 in the spreadsheet
It also had three corresponding for SENDING the data:
  • eyeballUpDown_stop (the same field name with _stop appended) -  data in E2 in the spreadsheet
  • eyeballLeftRight_stop -  data in  F2 in the spreadsheet
  • text _stop (this is the piece of text used to make Twitr_janus talk -  data in G2 in the spreadsheet
The difference between them the data in the entry cells A2-C2 and that in the sending cells E2-G2 by prefixing a control character to the contents of each field. The character is the same for each, and is based on the contents of field D2.

This character was there to be used by Processing to know where to parse the data in the RSS feed. The character can be any SINGLE character, but it must be something that is not going to to be present in the RSS feed output or in normal speech.
In this case the following character has been used: ¬ 

Making the spreadsheet publish to the web automatically and obtaining the RSS url

From the file menu, choose the "Publish to the Web" option. This make the data public as read only. You can still edit the original sheet and control share rights for edit, but the data is now available to use.
The checkbox must be checked for "Automatically republish when changes are made". This will force the RSS feed to refresh when you type in new data.

You then need to get the url for the RSS feed. Select RSS from the "Get a link to the published data" option, and copy the url.

What the RSS data looks like

The output of the RSS feed will be read as a string that looks like this:


How Processing extracts the data from the RSS feed

(You can download the entire Processing sketch for this particular version Twitr_janus  on Github: https://github.com/downloads/rosemarybeetle/Twitr-Janus/twitr_janus_processing_control_PC_2.pde)

The Processing sketch uses a function called getGssData (). Here it is, with some notes about the data extraction using parsing. I have not explained the rest of the function, nor the variable initialisations.

void getGssData ()
{
  // uses Google SpreadSheets API to get public tweets from twitr_janus_eyeballs published spreadsheet
gssTextCheck = gssText;

  println ("@@@");
  println ("[Start Inside printGSS]");
  println ();


  String [] texty = loadStrings(gssApiString);
The line above is using the Processing function 'loadStrings'. 'gssApiString' has been initialised earlier with the RSS feed URL (https://spreadsheets.google.com/feeds/list/0AgTXh43j7oFVdDJSaHU1ejFqdVRTZU1ZZ1Fabmt2UXc/od6/public/basic?alt=rss)
and loading the raw XML output as a string into an array (texty[])

  String [] texty2 = split (texty[0], '¬'); //  pulling out data with stop character
The line above is using the Processing function 'split' to break down the feed text based upon the special stop character '¬'. It is creating an array (texty2[]) which will have 5 values in it, because the feed has four of these characters in it, which are splitting the feed string into 5 sections.

The values are:
  • texty2[0] = contents of everything before the '¬' in cell D2
  • texty2[1] = contents of everything after that and before the '¬' in cell E2
  • texty2[2] = contents of everything after that and before the '¬' in cell  F2
  • texty2[3] =  contents of everything after that and before the '¬' in cell  G2
  • texty2[4] =  contents of everything after the '¬' in cell G2

  String [] texty3 = split (texty2[4], '<'); // get rid of trailing text after <
The line above finally gets  texty2[4] (= the contents of everything after the '¬' in cell G2) and splits it based on the '<' tag in XML, which is the character that follow the contents of G2. This leaves the text from cell G2 which is set as the variable gssText in the line below:
  gssText = texty3[0];

  gssTextLength= gssText.length();

The lines below do a similar thing to extract the contents out of E2
  // @@@@@@@@@@@@@@@@
  String [] texty4 = split (texty2[2], ',');
  gssEyeballUpDown = int (texty4 [0]);
  print ("gssEyeballUpDown = ");
  println (gssEyeballUpDown);
  println ();
Finally the lines below do a similar thing to extract the contents out of F2
  // @@@@@@@@@@@@@@@@
  String [] texty5 = split (texty2[3],',');
  gssEyeballLeftRight = int (texty5 [0]);
  print ("gssEyeballLeftRight = ");
  println (gssEyeballLeftRight);
  println ();
  // @@@@@@@@@@@@@@@@
  //print ("texty= ");
  //println (texty);
  println ();
  //print ("texty2[2]= ");
  //println (texty2[2]);
  println ();
  //print ("texty2[3]= ");
  //println (texty2[3]);
  println ();
  print ("gssText = ");
  println (gssText);
  println ();
  println ();
   // following lines return the contents of tweet check (last new tweet)

  println ("[End Inside getGSS]");
  println ("@@@");
  println ();

  if (gssText.equals(gssTextCheck)==false)
  {
    println ("inside GSS checking IF");
    print (gssText);
    println ("");
    print (gssTextCheck);
    println ("@");
    port.write(gssTextLength);
    tts.speak(gssText);
  };
}

Advantages of using Google spreadsheets to send data

Although this is a similar method as having Twitr_janus listen for tweets, using a spreadsheet has advantages, mainly because there is much more control over the RSS feed format than there is the Twitter API. This means:
  • Several fields of data can be changed in a spreadsheet simultaneously, by using multi-cell cut and paste. So if the left/right and up/down position of eyeballs needed to be changed simultaneously, this could be done in one paste action.
  • By prefixing the data in control fields with an unusual stop character, the Processing command that is parsing the text, will not get confused by common punctuation used in tweets. Notably commas and "quote marks", are used by the Twitter API as field delimiters, and these cause parsing issues truncating the message unexpectedly. The Google method eliminates this.
  • The maximum number of calls in a set time to Google spreadsheet RSS feeds appears to be much higher than Twitter's API. This allows Twitr_janus's brain to check much more frequently (an improvement from about every 28 seconds maximum with Twitter API, to every 5 seconds or faster with the Google feed) 
  • It is probably possible to write a javascript gadget that will write to Google spreadsheet cells in a more interactive way (such as an onscreen slider that writes values to the cells)
This took a lot of coffee and about 7 hours of brainache...

Download this version of Twitr_janus Processing sketch on Github: https://github.com/downloads/rosemarybeetle/Twitr-Janus/twitr_janus_processing_control_PC_2.pde

Sunday, 29 July 2012

Manipulating tweet character count in Arduino to control lip-sync duration

Simplified serial data connection between Processing and Arduino

Detail of the Processing sketch, showing sub-clause of tweetCheck () function which is checking incoming tweets from @twitr_janus. When it detects one, it sends a trigger signal to the Arduino to start jaw movement, then immediately initiates text-to-speech on the text. This will make the jaw move as the text speech audio plays.
Download the complete Processing sketch running on the PC detecting tweets
Download the complete Arduino sketch running on the Arduino board
Twitr_janus Processing and Arduino sketches

Simple example - triggering an Arduino response to a tweet by sending a flag over the serial port.

This clause checks if it is a new tweet:

if (tweetText.equals(tweethCheck) == false)  

"tweetText" is the latest value of last tweet from twitr_janus, "tweetCheck" is the last new tweet.
It is doing two things here. Firstly it is sending a message of value NULL to the serial port (called imaginatively "port") which will send it over to the USB connection to the arduino board. This could be done wirelessly in principle.

port.write("NULL") 

The NULL character is converted to 0 when it is transmitted as data over the serial connection. The Arduino will start a jaw movement control signal if it detects a signal of value 0. In the code shown, it is simply turning a PIN on and off 8 times, with a 100ms delay before switching between HIGH and LOW, to give a 200ms period..

Adding logic - using tweet length to control the lip-sync signal duration

Using a simple value is fine for an on-off trigger, but the lip-sync should only occur for the duration of the speech generated by the tweet. This will vary depending on the number of characters in the tweet. This can be obtained using the length() method.

tweetLength= tweetText.length();

By sending the tweet length (number of characters) instead of a simple trigger, lip-sync logic can be created in the Arduino sketch running on the board, that will cause the lip sync duration to match the speech audio duration.

The Arduino function is called jawChomper ().

"incomingByte" is the tweet length in number of characters
"chompFactor" is a scaling factor that reduces the character length into jaw movement number. This is because when you speak your jaw does not move for every character. It moves based on words, which are groups of characters.
"chompDelay" is the standard duration in milliseconds, of the alternate HIGH and LOW values of the control signal.
"chompRand" is a controlled randomisation factor. This is added, to cause each HIGH or LOW to vary in duration. This is because the code cannot determine word length, so the jaw cannot be synced to the words. Just turning the signal on and off is not good enough. The brain can easily detect a rigidly uniform rhythm and it will jar. Varying the duration of up and down motion is a simple way to trick the brain into thinking the jaw is in sync with the speech, when it is actually just stopping and start in an asymmetrical rhythm.

void jawChomper ()
{
  // this function sets jaw bighting rate for
    // read the incoming byte:
 
   digitalWrite(13, HIGH);
   for (int i = 0; i < (incomingByte/chompFactor); i++) {
    // turn the pin on:
    digitalWrite(peakPin, HIGH);
    delay(chompDelay+random(chompRand));                
    // turn the pin off:
    digitalWrite(peakPin, LOW);
     delay(chompDelay+random(chompRand));
   }
 
   digitalWrite(13, LOW);
   
    // say what you got:
    Serial.print("I received: ");
    Serial.print(incomingByte);
    Serial.print(incomingByte, DEC);
    }

The voltage signal sent from the Arduino (signal pin here is peakPin - pin 6 as it happens) is connected to the base of a transistor with a resistor to supply a signal current. When the pin is HIGH, the transistor will amplify it to a current high enough to trigger the relay and turn on the power to the motor (not shown here).
When the signal goes LOW, the transistor current will stop and turn of the relay.
Arduino and lip-sync relay circuit

Sunday, 22 July 2012

Twitr_janus talks through temporary animatronic jaw

Twitr_janus speaks and moves Finally, I have got the full loop through from sending a tweet to automatically make. Remote puppet talk and move its mouth in time as it speaks.

Twitter to speech converter kit list

  • Twitter app (here on iPad) to Internet
  • Internet to PC running Processing sketch periodically scans for twets and strips message
  • Processing converts to spoken output using tts.h text-to-speech library
  • Speech played through audio amplifier
  • Amplifier secondary speaker output connected to Arduino analog input
  • Voltage detection in Arduino triggers output pulses on peaks
  • Peaks switch power on and off, from model railway transformer connected to car door actuator motor
  • Actuator moves jaw in time to speech peaks


Twitr_janus audio out Twitr_janus amplifier audio connections
Twitr_janus model railway power unit and Arduino Twitr_janus relay circuit Twitr_janus Processing sketch Car door lock actuator driving cardboard mouth









































Watch the complete loop from tweet to talking cardboard...

Friday, 13 July 2012

Paper prototyping jaw opening mechanism


Just discovered car-door lock mechanism which is an awesomely useful device that will push quite a big load from a 12V pulse. This may be the answer to lyp-synched puppet mouth operation via Twitter-to-Text-to-Speech-to-Jaw-Motion!

Nothing like a good Rotring pen for scrawling out circuit ideas
Car door lock motor circuit






















Here's what the motor looks like (it's a bit of a crap photo, off Maplins

Thursday, 5 July 2012

Twitr_janus speaks...

Success!

A successful test of sending a tweet from device to another remote device over the web to make Twitr_janus talk

This allows speech puppetry to Twitr_janus anywhere in the world!
  • on controller device (an iPad here, but could be any other web device)
  • text sent as tweet to Twitr_janus account on Twitter to web to Twitter
  • receiver device from Twitter API to web
  • to Processing sketch where API is parsed to extract tweet text
  • text is converted to speech using guru.freetts library and Processing native text-to-specch (TTS)

ha ha ha ha...

Watch the test in action as it was filmed!

Wednesday, 27 June 2012

Twitr Janus has sight and hearing

Twitr Janus is a remote control puppet head.
Previously, the basic means by which to make it to speak was established. This will work by enabling it to listen (via a Processing sketch) to tweets from the @Twitr_janus account on Twitter. The tweets will eventually get converted into spoken sentences using text to speech.

and so onto hearing and vision...

To get sight requires an eye. In this case an HD video camera. a cheap one will do, but this was handy for testing. A lower res camera may actually be more desirable to keep a fast connection and make it reactive. This one has a decent microphone too, which is harder to find. It is likely that this will be used, but with the video streaming settings set low, but the audio high.
Twitr Janus and its eye

Various possible options were considered to enable streaming of video and audio from the remote head to a control PC elsewhere.
These included:

  • Google hangouts but this is aimed at socialising, and can't be easily activated remotely. 
  • running a mini server with a/v/ streaming, but I couldn't be bothered to work out the security/ firewall/port settings. 
  • a plug and play IP or ethernet webcam, but this was not proven and would need to be bought before testing.
In the end, after some trawling discussions, it occurred to me that Skype might work. This proved to be rather useful because:

  • it can be set up to make a video and audio streaming connection really easily
  • it can run in the background, and be activated remotely by calling it
  • security is handled by  the in-built privacy settings and by creating controlled accounts for Twitr Janus and a controller, which can only access calls by each other
  • once account access is restricted, the connection can be activated remotely by the controlling account by setting the slave account to automatically connect (set in Skype) 

Here is a prototype of the communication technology components, joined up to demo the following:

  • it is possible to start the video/audio link to enable Twitr Janus to see and hear (by calling on Skype - the slave account on Twitr Janus control PC would need to be in tray mode)
  • it is possible to respond remotely to someone talking to Twitr Janus by tweeting the response to the @Twitr_janus Twitter account which will eventually be converted to speech
and so...

The iPad (master):


  • the controlling device here. It also could be a phone or laptop as the Twitter and Skype services are web based (or app based if needed)

The laptop (slave control PC):



  • has the web cam attached pointing at the Arduino board.
  • has Skype running streaming video and audio from the webcam over the Internet to the remote master control (iPad here)
  • has a Processing sketch running which is listening to Twitter and sending data to the Arduino if a new tweet from @Twitr_janus has been sent

The arduino:

  • Has the Firmata for arduino library loaded, which is making the serial connection possible to the Processing script, and hence to Twitter
  • Has an LED indicating when a tweet has been received

very satisfying!!
Twitr Janus  remote communication with Arduino board

Close up of the eye watching the response LED indicating that a Tweet has been received!
Remote monitoring of Twitr Janus

Sunday, 24 June 2012

Twitr Janus is created

Twitr Janus icon

It lives!

I must stop watching films like Promethius...

Twitr Janus is a 2-way communication puppet with remote animatronic control built on common technologies (well, it will be eventually!)
When finished it will be a physical avatar that can be controlled remotely to interact with people over the Internet. 


Basic objectives for this project are:
  • to make a talking puppet head that can be manipulated remotely
  • to build a completed working puppet
  • as far as possible to build all this in open source software such as Arduino, Processing with communication using open data such as  APIs from common services such as Twitter and Google Docs 
This could take a while!
So far:
  • it is just an Arduino puppet brain but no physical form yet
  • it has a Twitter feed through which it can think and communicate @twitr_janus
  • It has a puppetmaster brain running  Processing on a master computer that can control it across a serial port connection 

A quick video showing the initial brain functions....

 
Twitr Janus tweet retrieving script in Processing




















Processing script that is returning tweets using the Twitter API and passing them to the arduino which has a Firmata sketch loaded on it as firmware.

Wednesday, 13 June 2012

Initial thoughts on making Twitter drive a remote talking head

This is a "design" and I use that term loosely, that is an attempt to make a real head speak out tweets, from anywhere in the world - a sort of remote Twitter-to-speech-convertor!
Twitter to speaking head convertor





























To make this work there are a number of challenges, which include:
  • Connecting the head to the web, ideally remotely using wireless
  • Once connected getting the last tweet from a specific Twitter account (the control account)
  • Converting the tweet into speech
  • Making the speech signal animate a mouth to simulate talking
This is visualised above!

The step for getting the last tweet from a twitter account
Step one of this I appear to have solved in Processing (which is Java-based), which should mean it works in arduino.  

Here's a snap shot of the function and the output that it can produce.
It's calling the Twitter API, then parsing it and splitting it to create an array, then calling one element of that array, which is the last Tweet from twitter.com/rosemarybeetle