Twitr Janus has sight and hearing

Twitr Janus is a remote control puppet head.
Previously, the basic means by which to make it to speak was established. This will work by enabling it to listen (via a Processing sketch) to tweets from the @Twitr_janus account on Twitter. The tweets will eventually get converted into spoken sentences using text to speech.

and so onto hearing and vision...

To get sight requires an eye. In this case an HD video camera. a cheap one will do, but this was handy for testing. A lower res camera may actually be more desirable to keep a fast connection and make it reactive. This one has a decent microphone too, which is harder to find. It is likely that this will be used, but with the video streaming settings set low, but the audio high.


Various possible options were considered to enable streaming of video and audio from the remote head to a control PC elsewhere.
These included:

• Google hangouts but this is aimed at socialising, and can't be easily activated remotely. 
• running a mini server with a/v/ streaming, but I couldn't be bothered to work out the security/ firewall/port settings. 
• a plug and play IP or ethernet webcam, but this was not proven and would need to be bought before testing.

In the end, after some trawling discussions, it occurred to me that Skype might work. This proved to be rather useful because:

• it can be set up to make a video and audio streaming connection really easily
• it can run in the background, and be activated remotely by calling it
• security is handled by  the in-built privacy settings and by creating controlled accounts for Twitr Janus and a controller, which can only access calls by each other
• once account access is restricted, the connection can be activated remotely by the controlling account by setting the slave account to automatically connect (set in Skype) 

Here is a prototype of the communication technology components, joined up to demo the following:

• it is possible to start the video/audio link to enable Twitr Janus to see and hear (by calling on Skype - the slave account on Twitr Janus control PC would need to be in tray mode)
• it is possible to respond remotely to someone talking to Twitr Janus by tweeting the response to the @Twitr_janus Twitter account which will eventually be converted to speech

and so...

The iPad (master):

• the controlling device here. It also could be a phone or laptop as the Twitter and Skype services are web based (or app based if needed)

The laptop (slave control PC):

• has the web cam attached pointing at the Arduino board.
• has Skype running streaming video and audio from the webcam over the Internet to the remote master control (iPad here)
• has a Processing sketch running which is listening to Twitter and sending data to the Arduino if a new tweet from @Twitr_janus has been sent

The arduino:

• Has the Firmata for arduino library loaded, which is making the serial connection possible to the Processing script, and hence to Twitter
• Has an LED indicating when a tweet has been received

very satisfying!!


Close up of the eye watching the response LED indicating that a Tweet has been received!

Twitr Janus is created

It lives!

I must stop watching films like Promethius...

Twitr Janus is a 2-way communication puppet with remote animatronic control built on common technologies (well, it will be eventually!)
When finished it will be a physical avatar that can be controlled remotely to interact with people over the Internet. 


Basic objectives for this project are:

• to make a talking puppet head that can be manipulated remotely
• to build a completed working puppet
• as far as possible to build all this in open source software such as Arduino, Processing with communication using open data such as  APIs from common services such as Twitter and Google Docs 

This could take a while!
So far:

• it is just an Arduino puppet brain but no physical form yet
• it has a Twitter feed through which it can think and communicate @twitr_janus
• It has a puppetmaster brain running  Processing on a master computer that can control it across a serial port connection 

---

A quick video showing the initial brain functions....

 

Processing script that is returning tweets using the Twitter API and passing them to the arduino which has a Firmata sketch loaded on it as firmware.

Initial thoughts on making Twitter drive a remote talking head

This is a "design" and I use that term loosely, that is an attempt to make a real head speak out tweets, from anywhere in the world - a sort of remote Twitter-to-speech-convertor!






























To make this work there are a number of challenges, which include:

• Connecting the head to the web, ideally remotely using wireless
• Once connected getting the last tweet from a specific Twitter account (the control account)
• Converting the tweet into speech
• Making the speech signal animate a mouth to simulate talking

This is visualised above!

The step for getting the last tweet from a twitter account

Step one of this I appear to have solved in Processing (which is Java-based), which should mean it works in arduino.  

Here's a snap shot of the function and the output that it can produce.

It's calling the Twitter API, then parsing it and splitting it to create an array, then calling one element of that array, which is the last Tweet from twitter.com/rosemarybeetle

For details on the Twitter API, see https://dev.twitter.com/docs/api/1/get/statuses/user_timeline

Making an animated GIF from a pen and ink drawing

Cell-animated eye

A single cell grows into an eye.In the blink of itself, it’s extinct. See below for how this was made.

This was hand drawn using pen and ink.

Here is the original concept sketch. Pretty simple. Because this was for an animated GIF, it wanted to have a looping repeat feel to it, and have a beginning and end that worked together

I did not want to actually hand-animate this - far too long-winded. Instead the basic units of the drawing that would need to be animated separately were considered, and these were drawn more carefully using pen and ink.

 

These were then photographed with a digital camera and imported in Flash to manipulate them. Within Flash teh "Trace bitmap" function is used to create vectors from the imported image. The white areas are deleted, leaving just the ink lines, which can then be treated as any other shape in Flash. E.g. the blue iris colouring was added in Flash using a gradient fill.

It gets quite complex in Flash, but essentially each "thing" needs it's own layer, so it can be animated separately from other parts. This screenshot is a bit rubbish! 

Once I was happy I had recreated the design into a moving animation, it was exported using "Export Movie > animated GIF"

Animated GIF, made to upload to a public submissions space on the website for The Photographer's Gallery's Born in 1987: The Animated GIF exhibition.
http://joyofgif.tumblr.com/

HYPNO-PUG

HYPNO-PUG

Small, daft, jolly, stinky, grunty, lovely, friendly, dog.
Now available as an infinite loop self-hypnosis aide.

STARE DEEP INTO THIS GIF and feel the pug vibe evaporate your stresses.

TIP - look at the dog, but concentrate on the leaves.

Animated GIF, made to upload to a public submissions space on the website for The Photographer's Gallery's Born in 1987: The Animated GIF exhibition.

How this was made

This was created by importing an iPhone video of the pug into Flash, then drawing a large rectangular shape on a layer over it. The rectangle was created as a symbol in Flash, with it's alpha (transparency) value set to about 17% so you could still see the video underneath it. The colour of the rectangle was varied along the animation time using motion tweening to give the varying colorisation effect.

The 'Export as Movie' process in flash was used with the animated GIF setting.