Wednesday 6 January 2010

Peer to peer video and audio streaming with Flex

Hi Readers,

This going to be a quick blogpost about how to build a simple video and audio streaming app in Flex. All the fame is for the Adobe Stratus service, and for Tom Krcha, a very great Adobe tech evangelist: http://www.flashrealtime.com/basics-of-p2p-in-flash/ (this is the original article.) Most of my code is from there! Great thanks for your excellent tutorial.

So, first take a look at the Stratus service: http://labs.adobe.com/technologies/stratus/ It allows to your Flash app to create P2P connection between clients. Cool, isn't it?

Our app will open the default camera and microphone streams and transfer to any other client we share our peerID with. I succeeded to open 3 additional connection. That's already two more than Skype allows. Of course, it's not that good.



Ok ok, maybe it's better to try it out: Multivideo.swf
- Allow your camera and microphone to connect
- Share your peerID with your partners using the app (your peerID is in the first text box)
- Add new streams by adding other contacts' peerIDs: copy their peerID into the 2nd textbox and click on [Add contact]
(On some systems sometimes firewall blocks the stream.)

Let's do it than. We need some UIComponent. A base video display and some textbox and a button:
<mx:HBox id="video_stack" top="10" left="10">
<mx:VBox>
<mx:VideoDisplay id="my_video_display" width="320" height="240"/>
<mx:HBox>
<mx:TextInput width="320" id="farPeerId_text" text="Your Peer ID is loading..."/>
</mx:HBox>
<mx:HBox id="add_contact_container" visible="false">
<mx:TextInput id="contact_peer_id_text" width="200"/>
<mx:Button label="Add contact" click="{addContact();}"/>
</mx:HBox>
</mx:VBox>
</mx:HBox>

HBox allows us just pushing the new stream UI elements into the horizontal queue.
Using Stratus we have to register for a Stratus API key. We will get an ID and the url:
private var rtmfpServer:String = 'rtmfp://stratus.adobe.com/cbd2224f9a56771b3d4d05c3-bd9b549abca2';

Than make some vars:
private var nc:NetConnection;
private var sendNS:NetStream;
private var neerPeerID:String;

private var cam:Camera;
private var mic:Microphone;

When the application successfully loaded, we can init our camera, microphone and the net connection for outgoing streams:
private function init():void {
initCamera();
initNetConnection();
}

Taking the cam/mic is quite straightforward:
private function initCamera():void {
if (Camera.names.length > 0) {
cam = Camera.getCamera();
my_video_display.attachCamera(cam);
}

if (Microphone.names.length > 0) {
mic = Microphone.getMicrophone();
}
}

I said so. Than the net connection to the stratus server through RTMFP protocol:
private function initNetConnection():void {
nc = new NetConnection();
nc.addEventListener(NetStatusEvent.NET_STATUS, netStatusEvent);
nc.connect(rtmfpServer);
}

When the net connection is ready, we can establish the sending stream:
private function netStatusEvent(event:NetStatusEvent):void {
trace('NetConnection status event (1): ' + event.info.code);
if (event.info.code == 'NetConnection.Connect.Success') {
neerPeerID = nc.nearID;
farPeerId_text.text = neerPeerID;
initSendNetStream();
add_contact_container.visible = true;
}
}

This is the point we've got out peerID we can share. wOOt! On setting the provider stream we can attach out camera to the stream. And we can define a client object accepts the connection (return true;):
private function initSendNetStream():void {
sendNS = new NetStream(nc, NetStream.DIRECT_CONNECTIONS);
sendNS.addEventListener(NetStatusEvent.NET_STATUS, netStatusEvent);

var clientObject:Object = new Object();
clientObject.onPeerConnect = function(ns:NetStream):Boolean {return true;}

sendNS.client = clientObject;
sendNS.attachCamera(cam);
sendNS.attachAudio(mic);
sendNS.publish('video');
}

So far we have a working camera, microphone, and the stream is already on. Now prepare for accepting others stream:
private function addContact():void {
var nc2:NetConnection = new NetConnection();
nc2.addEventListener(NetStatusEvent.NET_STATUS, function (event:NetStatusEvent):void {
trace('NetConnection status event (2): ' + event.info.code);
var receiveNS:NetStream = new NetStream(nc2, contact_peer_id_text.text);
receiveNS.addEventListener(NetStatusEvent.NET_STATUS, netStatusEvent);
receiveNS.play('video');

var video:Video = new Video();
video.attachNetStream(receiveNS);

var uic:UIComponent = new UIComponent();
uic.width = 320;
uic.height = 240;
uic.addChild(video);
video_stack.addChild(uic);

contact_peer_id_text.text = '';
});
nc2.connect(rtmfpServer);
}

When we get a new stream, we create a new video object and add onto the stage.
I know, it's far far basic, but it's good for starting a tool like this.

Download the source code.

Bests,
Peter

29 comments:

  1. This comment has been removed by a blog administrator.

    ReplyDelete
  2. This comment has been removed by a blog administrator.

    ReplyDelete
  3. I used Flex SDK 3.5. You also have to set the default Flash Player version to 10.0.0.

    ReplyDelete
  4. Hello Peter,
    thank you for this example, it ran at me at once. I have a question about how I can improve the quality of the camera?
    --------------------------------------------
    private function initCamera():void {
    if (Camera.names.length > 0) {
    cam = Camera.getCamera();
    cam.setMode(320,240,30);
    my_video_display.attachCamera(cam);
    }
    ---------------------------------------------
    Unfortunately, this entry will change only my own video!
    All other videos (right) remain in tha old poor video quality!

    There is even the possibility of combining the ID with names?

    Thank you and am looking forward to your reply.

    Greetings from Hamburg / Germany

    Fiedrich Koch

    ReplyDelete
  5. Hi Friedrich,

    Unfortunately I'm not aware of the technical details of streaming (yet). I assumed, if you change the input source's density to a higher resolution, you will get a better view on the streamed side. I think you checked that already. It can be a bandwidth issue or a NetStream config? I don't know.
    The other, using names instead of ID-s: you can do tricks, but PeerID is the standard way to identify clients. You need them. But I'm afraid if you want to do it in a user friendly way, you need some kind of backend store to keep track of live clients and their name's.

    Bests,
    Peter (from Szeged/Hungary ;) )

    ReplyDelete
  6. Its not working, can you explain?

    [sun@example src]$ /home/sun/Downloads/flex/sdks/3.0.0/bin/mxmlc -target-player=10 flashs.mxml
    Loading configuration file /home/sun/Downloads/flex/sdks/3.0.0/frameworks/flex-config.xml
    /var/www/html/flash/src/flashs.mxml(43): Error: Access of possibly undefined property nearID through a reference with static type flash.net:NetConnection.

    neerPeerID = nc.nearID;

    /var/www/html/flash/src/flashs.mxml(51): Error: Incorrect number of arguments. Expected no more than 1.

    sendNS = new NetStream(nc, NetStream.DIRECT_CONNECTIONS);

    /var/www/html/flash/src/flashs.mxml(51): Error: Access of possibly undefined property DIRECT_CONNECTIONS through a reference with static type Class.

    sendNS = new NetStream(nc, NetStream.DIRECT_CONNECTIONS);

    /var/www/html/flash/src/flashs.mxml(67): Error: Incorrect number of arguments. Expected no more than 1.

    var receiveNS:NetStream = new NetStream(nc2, contact_peer_id_text.text);

    [sun@example src]$

    ReplyDelete
  7. Then i tried Flex SDK 3.5 still same result:

    [sun@example bin]$ ./mxmlc /var/www/html/flash/src/flashs.mxml
    Loading configuration file /home/sun/Downloads/flex3.5/frameworks/flex-config.xml
    /var/www/html/flash/src/flashs.mxml(43): Error: Access of possibly undefined property nearID through a reference with static type flash.net:NetConnection.

    neerPeerID = nc.nearID;

    /var/www/html/flash/src/flashs.mxml(51): Error: Incorrect number of arguments. Expected no more than 1.

    sendNS = new NetStream(nc, NetStream.DIRECT_CONNECTIONS);

    /var/www/html/flash/src/flashs.mxml(51): Error: Access of possibly undefined property DIRECT_CONNECTIONS through a reference with static type Class.

    sendNS = new NetStream(nc, NetStream.DIRECT_CONNECTIONS);

    /var/www/html/flash/src/flashs.mxml(67): Error: Incorrect number of arguments. Expected no more than 1.

    var receiveNS:NetStream = new NetStream(nc2, contact_peer_id_text.text);

    [sun@example bin]$

    ReplyDelete
  8. Finally got it working. Here is a summary Flex SDK < 3.5 won't work + Compile Mxml file using -target-player=10

    Here is successfull compile report:

    [sun@example bin]$ ./mxmlc -target-player=10 /var/www/html/flash/src/flashs.mxml
    Loading configuration file /home/sun/Downloads/flex3.5/frameworks/flex-config.xml
    /var/www/html/flash/src/flashs.swf (196791 bytes)
    [sun@example bin]$

    ReplyDelete
  9. Hi there,

    We are interactive media students and we're working on something similar like this blogpost. Unfortunately the source code isn't working anymore!

    Could you please e-mail me the source code to rhartog [at] gmail dot com?

    Thanks!

    ReplyDelete
  10. The Multivideo.swf and the downloadable source code file's links are corrected. Thanks!

    ReplyDelete
  11. Hi there,
    I've been trying to use your code above, in Flash CS5. I'm having an issue when I try to add a contact. I get an initial success message but this is immediately following by a closed message. Would you have any idea why this might happen?
    Thanks

    ReplyDelete
  12. Hm, good question. I'd check if the stratus api key is valid. Connections are not blocked by firewalls. Than checking the error message thoroughly.
    What is the exact error message btw?
    My code is in Flex, your's is in Flash, so maybe it has some differences.
    If you can't get through of it, can you send me code, or some details?

    ReplyDelete
  13. hi itarato, could you write down an example or any clue how to add to the appication a text chat ?

    ReplyDelete
  14. I'd try NetStream.publish() with simple strings:
    http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/NetStream.html#publish()

    ReplyDelete
  15. how can we set the VideoDisplay as resizable component?

    ReplyDelete
  16. Well, I'm not sure. I've never tried it yet. I would play with the container element and make that resizable. If you think of the video resolution, that I've no idea. Maybe you have to redefine the stream...

    ReplyDelete
  17. Hi itarato,
    can you tell me please if the stream passes directly between the two users or it passes by the server to the which case it will use the band-width of the server ?

    Thanks in advance.

    ReplyDelete
  18. Hi Hicham,

    As far as I know (by the desc http://labs.adobe.com/technologies/cirrus/) the Straus (now it's called Cirrus) server only helps establishing the communication but then it's happening directly peer to peer between flash player clients. But it's better to ask somebody who used it recently.

    Peter

    ReplyDelete
  19. Is there any chance to port this to android? Or could I just start a new flex project and run in on my android device?

    ReplyDelete
    Replies
    1. Hi Tobias,

      I haven't done any Android dev, but as far as I know you can do that in Flex if the video is supported a similar/same way.

      Delete
  20. Is there any way to achieve the same functionality by using just Flash Professional CS5? I'm fairly unfamiliar with Flash Builder's interface and Flex.

    (Thank you for the tutorial, greetings from Kecskemét / Hungary :D)

    ReplyDelete
    Replies
    1. Hi suexID,

      Haha, from where I was born :) Cool.
      I think you can do that, yes. There is nothing here that is not available in the standar AS3 library. I'm only using Flex because that's a proper dev IDE.

      Delete
    2. Thank you for your reply, I'll try to implement it then.

      Delete
  21. Itarato, thank you for this informative blog. Wanted to report one issue to see if it happens only to me. The local video streaming with Multivideo.swf works on Ubuntu Linux, but not on MacOS X.

    Environment:
    - Multivideo.swf, compiled with Flex 4.6
    - Ubuntu Linux (11.10) & Chrome & Flash (11.2.202.235)
    - MacOS X (10.6.8) & Safari & Flash (11.2.202.235)

    Steps for testing the local video rtmfp streaming.
    - Open the Multivideo.swf with the browser
    - Copy and paste the nearID to the text box
    - Press "Add Contact"
    - On Ubuntu Linux, a new window appears and shows the mirrored image of the camera input.
    - On MacOS X, Camera input works, but new windows does not show up.

    Thanks for your time.

    ReplyDelete
  22. Hi Domaemon,

    Nice catch! I don't know. I've tried and same happens here too. In Chrome and Safari too. For a moment I thought somebody blocks the camera, but in fact it's not the camera, it's a stream input. So should be fine. The window's scrollbar is growing, it means it adds something. Also, if works in Ubuntu it means it cannot be a global issue.
    I'm sorry I cannot think of any helpful. Anybody, ideas?

    ReplyDelete
  23. Itarato, thank you for your reply. Good to know that the error is reproducible. I assume Adobe Forum is the right place to forward this question now.

    ReplyDelete
  24. Tahnk you for this excellent tuto, but i have a problem with this code.
    When I try it, I can send a stream but the computer wich send can't receive the stream of the other computer, do you know how can I solve this please ?

    ReplyDelete

Note: only a member of this blog may post a comment.