Twitr Janus has sight and hearing

Twitr Janus is a remote control puppet head.
Previously, the basic means by which to make it to speak was established. This will work by enabling it to listen (via a Processing sketch) to tweets from the @Twitr_janus account on Twitter. The tweets will eventually get converted into spoken sentences using text to speech.

and so onto hearing and vision...

To get sight requires an eye. In this case an HD video camera. a cheap one will do, but this was handy for testing. A lower res camera may actually be more desirable to keep a fast connection and make it reactive. This one has a decent microphone too, which is harder to find. It is likely that this will be used, but with the video streaming settings set low, but the audio high.


Various possible options were considered to enable streaming of video and audio from the remote head to a control PC elsewhere.
These included:

• Google hangouts but this is aimed at socialising, and can't be easily activated remotely. 
• running a mini server with a/v/ streaming, but I couldn't be bothered to work out the security/ firewall/port settings. 
• a plug and play IP or ethernet webcam, but this was not proven and would need to be bought before testing.

In the end, after some trawling discussions, it occurred to me that Skype might work. This proved to be rather useful because:

• it can be set up to make a video and audio streaming connection really easily
• it can run in the background, and be activated remotely by calling it
• security is handled by  the in-built privacy settings and by creating controlled accounts for Twitr Janus and a controller, which can only access calls by each other
• once account access is restricted, the connection can be activated remotely by the controlling account by setting the slave account to automatically connect (set in Skype) 

Here is a prototype of the communication technology components, joined up to demo the following:

• it is possible to start the video/audio link to enable Twitr Janus to see and hear (by calling on Skype - the slave account on Twitr Janus control PC would need to be in tray mode)
• it is possible to respond remotely to someone talking to Twitr Janus by tweeting the response to the @Twitr_janus Twitter account which will eventually be converted to speech

and so...

The iPad (master):

• the controlling device here. It also could be a phone or laptop as the Twitter and Skype services are web based (or app based if needed)

The laptop (slave control PC):

• has the web cam attached pointing at the Arduino board.
• has Skype running streaming video and audio from the webcam over the Internet to the remote master control (iPad here)
• has a Processing sketch running which is listening to Twitter and sending data to the Arduino if a new tweet from @Twitr_janus has been sent

The arduino:

• Has the Firmata for arduino library loaded, which is making the serial connection possible to the Processing script, and hence to Twitter
• Has an LED indicating when a tweet has been received

very satisfying!!


Close up of the eye watching the response LED indicating that a Tweet has been received!