Live.pics.io — How We Developed a Service Using WebRTC

Pics.io Team -

Six weeks ago we launched Live.pics.io, a free web-service to share photos and images the new way. The technology part of it is extremely interesting because it leverages one of the newest web-technologies (Web Real-Time Communications). This post tells about technical challenges we overcome during Live development.

Live.pics.io recreates the experience of real-world slideshows, when your guests would come by to see photos and hear you tell the story behind the shots. Developing our main product Pics.io we have an expertise in working with different raw formats in a browser. It allows owners of Canon and Nikon to skip the step of Raw-conversion and show photos immediately after shooting. Other cameras require conversion to medium DNG format.

If you’re not a tech enthusiast I recommend scrolling this post to the bottom. Some hardcore geeky story is about to start.

A few words about the technology

In fact, WebRTC is very similar to websockets. A little bit different. Just a bit. We need to transmit images and sound. Let’s take RTCPeerConnection to connect peers, MediaStream to broadcast audio and RTCDataChannel to transfer pictures. Yet, to make this work we’re gonna need some server-side to connect peers and transmit controlling instructions. But more on that later.

WebRTC via PeerJS

We decided that to deploy this fast we should use well-tested solution. After studying several open source projects that wrap WebRTC we chose PeerJS: because of the high rating on GitHub and the advice of Lvivsky, who has got a hell of experience creating Speaker library using Dart / WebRTC.

PeerJS is a wrapper for WebRTC which allows to forget about its innards. Eventually we had to digg into its source when discovered the limit for file transferring of ?6MB (RTCDataChannel). In general, the library has left a pleasant experience, and maintainers of the project helped us in a timely manner.

The first impression about PeerJS was amazing: one line of code on a server-side, the great interface on the client-side, support for Media calls (audio / video) and DataConnections. And, best of all, PeerJS allows to deal with the limit for data transmitting through DataConnections in Chrome 31 (1100 bytes). But every day is not Sunday and RTCDataChannel started failing with absolutely uninformative error-message Uncaught SyntaxError: An invalid or illegal string was specified. It took some coffee and a chat with contributors of PeerJS to find out that Chrome restricts the data transferring to 60kb/s.

NAT NAT NAT

Release day was coming. And it never comes without surprises. While we were on the LAN everything worked perfectly (MediaStream, DataChannel). But when someone was connected from outside the office began troubles. The sound, the data did not reach to the guests.

We have identified a few suspects:

  • DataChannel — which was the connector between all peers
  • Some bug that was difficult to reproduce
  • unable to reach through NAT (Network Address Translation that happens inside routing device)

Started adding Signaling server to connect peers, but this did not solve the problem. Searching for a bug also left us nothing. We found a lot, but it did not help.

“Probably the issue is connected to NAT” — now we were right. It’s strange, but the answer to this issue isn’t described really well in the documentation. Most libraries use the default servers. Unfortunately, it wasn’t enough and we had to search for additional servers . We ended with a huge config for ICEServers that can be found here. After that we successfully reached through the wall of NAT and started hearing voices of people from other sub-nets, cities and countries.

Technology challenges

Cross-browser compatibility

At the moment there is no way to make calls between different browsers. Firefox and Chrome are actively moving in this direction, but we’ll receive this support after a couple of versions.

Reaching through NAT

STUN and TURN servers is perhaps the main issue. Constantly testing, we noticed that within the same network everyone heard each other perfectly and received the data, but the people who were sitting behind the routers experienced persistent problems.

Conclusion

  • data transmission in PeerJS is limited to 60kb/s
  • to reach through NAT you need a bunch of STUN and TURN servers
  • don’t forget to note the scheme of you URI
  • WebRTC is gonna take the world eventually
  • WebRTC developers hate how they sound
  • non-stop coding is harmful for your health

The future of the service

We’re really happy to see that service is helpful and more and more people discover it through social networks and blogs. Visiting Live everyday you show us that it’s a handy service worth time of the developers. As underlying WebRTC technology matures, it’ll become more reliable too.