that provides browsers and mobile applications with Real-Time Communications (RTC) capabilities via simple APIs. WebRTC is a free, open project Webrtc is a cross platform solution with RTC capabilities. One can stream his own video stream be it from camera or screen recording or any other video to any peer via webrtc. If one is developing a real time peer to peer game with real time data sharing between the peers, webrtc is one of the options. Let’s understand how video calling from one peer to another works in webrtc. Getting started Since September 2017 Google started to distributed precompiled versions of webrtc for android via maven. If you want to play with the source and compile it yourself you can do it easily as per (Earlier you could compile it only on linux but now all the three major OS are supported). To use the pre compiled version simply add the following dependency. these steps compile 'org.webrtc:google-webrtc:1.0.22379' How does webrtc work? Before we start exchanging data between two peers via webrtc, we need to provide info to the peers about each other for media format negotiation and discovery. This is done via following protocols. is used for connecting peer to peer. is used to provide the metadata of the media content like resolution, encoding, bitrate, etc. If the two peers are not on same network then we will need to provide a server to provide the public address of the peers. If any of the network is firewall protected the we need to provide servers also. You can learn more about these protocols The mechanism to exchange ICE and SDP between the peers is called Signalling System and is usually done via websockets. Interactive Connectivity Establishment (ICE) Session Description Protocol (SDP) Session Traversal Utilities for NAT (STUN) Traversal Using Relays around NAT (TURN) here. Webrtc android API Most of our webrtc code will make use of and apis. is the equivalent of RTCPeerConnection in web world and is used to establish peer to peer connection. is used to create and objects_._ PeerConnection PeerConnectionFactory PeerConnection PeerConnectionFactory PeerConnection, MediaStream MediaStreamTrack ** **Before creating the factory object we need to initialize webrtc. As you can here provides options for enabling/disabling hardware accelerations(I had to disable in on certain devices) and setting field trials. are for enabling experimental webrtc features. Creating PeerConnectionFactory InitializationOptions Field trials val fieldTrials = (PeerConnectionFactory. + "/" + PeerConnectionFactory. + "/")val options = InitializationOptions.builder(application).setFieldTrials(fieldTrials).setEnableVideoHwAcceleration(videoAccelerationEnabled).createInitializationOptions()PeerConnectionFactory.initialize(options)factory = PeerConnectionFactory(PeerConnectionFactory.Options())val rootEglBase = EglBase.create()factory?.setVideoHwAccelerationOptions(rootEglBase. , rootEglBase. ) VIDEO_FRAME_EMIT_TRIAL TRIAL_ENABLED eglBaseContext eglBaseContext ** **Once we have the object we can now create a object which has and associated with it. Creating Media Streams PeerConnectionFactory MediaStream AudioTrack VideoTrack val localMediaStream = factory.createLocalMediaStream(MEDIA_ID)val audioSource = factory.createAudioSource(MediaConstraints())val audioTrack = factory.createAudioTrack(AUDIO_ID, audioSource)localMediaStream.addTrack(audioTrack) videoCapturer = if (Build.VERSION. > Build.VERSION_CODES. ) {createCameraCapturer(Camera2Enumerator(application))} else {createCameraCapturer(Camera1Enumerator(videoAccelerationEnabled))}val videoTrack = factory.createVideoTrack("VideoTrack", factory.createVideoSource(videoCapturer))localMediaStream.addTrack(videoTrack) SDK_INT LOLLIPOP Webrtc provides us a very easy way to use and API depending on the support. On supported devices we can use either of the APIs CameraVideoCapturer Camera Camera2 private fun createCameraCapturer(enumerator: CameraEnumerator): CameraVideoCapturer? {val deviceNames = enumerator._deviceNames_ for (deviceName in deviceNames) {if (enumerator.isFrontFacing(deviceName)) {val videoCapturer = enumerator.createCapturer(deviceName, null)if (videoCapturer != null) {return videoCapturer}}} for (deviceName in deviceNames) {if (!enumerator.isFrontFacing(deviceName)) {Timber.d("Creating other camera capturer.")val videoCapturer = enumerator.createCapturer(deviceName, null)if (videoCapturer != null) {return videoCapturer}}}return null} ** **Since we have the local ready, we need to render it on a view so that its visible to the user. is a in webrtc libray that does the rendering of webrtc frames for us. We can directly add it in layout xml. SurfaceViewRenderer MediaStream SurfaceViewRenderer View localViewRenderer.init(rootEglBase.getEglBaseContext(), null)localViewRenderer.setEnableHardwareScaler(true)localViewRenderer.setMirror(true)localViewRenderer.setScalingType(ScalingType.SCALE_ASPECT_FILL)val localVideoRenderer = VideoRenderer(localViewRenderer)videoTrack.addRenderer(localVideoRenderer) We have everything ready to transmit now, its time to create the We pass in a PeerConnection#Observer instance in the factory method, which is notified when a ICE candidate is generated (we need to send that to the other peer). The observer is also notified when the remote is available. we will attach a renderer to it just like the local . It also requires a list IceServer(STUN and TURN servers) which can be empty if testing on local network PeerConnection PeerConnection. MediaStream MediaStream val peerConnectionObserver = object : PeerConnection.Observer {override fun onIceCandidate(iceCandidate: IceCandidate) {localIceCandidatesSource.onNext(iceCandidate)}override fun onAddStream(mediaStream: MediaStream) {mediaStream.addRenderer(remoteRenderer)}...}peerConnection = factory?.createPeerConnection(getIceServers(), peerConnectionObserver) In Observer’s onIceCandidate we shouldn’t add the IceCandidate to PeerConnection (peerConnection.addIceCandidate(iceCandidate) till we have the remote SDP NOTE: Once the PeerConnection is created we need to initiate video call. The peer which initiates the call will and set its local SDP ( ). When the local SDP is set it will send that SDP to the other peer which will set its remote SDP ( ). Once the remote SDP is set it will with that sdp and set its local sdp when asnwer is created and send the local sdp to the peer which created the offer. The initiator peer will now set its remote SDP. CreateOffer/CreateAnswer createOffer peerConnection#setLocalSdp peerConnection#setRemoteSdp createAnswer fun createOffer() {peerConnection.createOffer(object : SdpObserver {override fun onCreateSuccess(sdp: SessionDescription) {setLocalSdp(sdp)}...}, getPeerConnectionConstraints())}fun setLocalSdp(sdp: SessionDescripton) {peerConnection.setLocalDescription(object : SdpObserver {override fun onSetSuccess() {api.sendSdp(peerConnection. )drainIceCandidates()}...}, sdp)} localDescription fun onOffer(sdp: SessionDescription) {peerConnection.setRemoteDescription(object : SdpObserver {override fun onSetSuccess() {createAnswer()}}, sdp)} fun createAnswer() {peerConnection?.createAnswer(object : SdpObserver {override fun onCreateSuccess(sdp: SessionDescription) {setLocalSdp(sdp)}}, getPeerConnectionConstraints())} ** **When the peers decide to end the connection its pretty important to do clean up in particular order as the C layer does a reference count check and will crash with assertion failures if the objects are not properly disposed. Dispose/ Clean up fun cleanUp() {peerConnection.dispose()viceoCapturer.dispose()videoSource.dispose()factory.dispose()localVideoRenderer.dispose()remoteRenderer.dispose()localViewRenderer.release()remoteViewRenderer.release()rootEglBase.release()} **Next Up**Basic socket implementation for webrtc using and video filters for webrtc. ktor.io References https://tech.appear.in/2015/05/25/Introduction-to-WebRTC-on-Android/ https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Signaling_and_video_calling
