What is video API Video API is an interface dedicated to providing audio and video transmission services, mainly divided into two types: and . static video API live video API Static Video API Static Video API is an API that provides video file play services. Service providers offer video file cloud storage and distribution services and provide video services through protocol interfaces such as HTTP and . CDN RTMP For example, YouTube and Instagram use Static Video API. Live Video API For example, , Yalla, and Uplive use the Live Video API. Live.me The static video API is easy to understand and pulls video files from the cloud through the streaming protocol. The live video API is more complicated. How does it ensure the video can be transmitted to the other end quickly, smoothly, and clearly? In this article, we will introduce the logic behind the live video API in detail. What Can The Video API Do The live video API application is becoming more extensive with the continuous improvement of network bandwidth and device performance. It makes many scenarios possible, such as: Live Online education Video conferencing Telemedicine Video calls Multiplayer What Happens After The Video API For the live video API, it is necessary to ensure that the video data can complete the end-to-end transmission within 500ms and, simultaneously, guarantee the clarity of the video picture and the flow of the call. Therefore, the live video API needs to ensure that the audio and video data can realize end-to-end data transmission in a fast, large and stable manner. A complex system is required to ensure the availability of the live video API. As shown in the figure, the live video API mainly layered the functions of 6 modules: 1. Audio and video capture Audio and video capture is the source of audio and video data collected via cameras, microphones, screens, video files, recording files, and other channels. It involves using color spaces, such as and and extracting audio features, such as sampling rate, number of channels, bit rate, audio frame, etc. RGB YUV 2. Audio and video preprocessing The audio and video preprocessing are mainly for business scenarios, and the collected data is processed once and again to optimize the customer experience. For example: Video data processing for beauty, filters, special effects, etc. Audio data processing includes voice change, reverberation, echo cancellation, noise suppression, volume gain, etc. 3. Audio and video encoding Audio and video coding ensures that audio and video data can be transmitted quickly and safely on the network. Commonly used encoding formats are Video encoding format: , Audio coding format: OPUS, AAC. H264 H265 4. Audio and video transmission Audio and video transmission is the most complex module in the video API. To ensure that the audio and video data can be transmitted to the opposite end quickly, stably, and with high quality in a complex network environment, streaming protocols such as RTP, HTTP-FLV, HLS, and RTMP can be used. Various anomalies such as data redundancy, loss, out-of-order, flow control, adaptive frame rate, resolution, jitter buffering, etc., must be resolved using multiple algorithms. So whether video API is worth choosing, we need to focus on whether the manufacturer has outstanding audio and video transmission advantages. 5. Audio and video decoding Audio and video decoding means that after the audio and video data is transmitted to the receiving end, the receiving end needs to decode the data according to the received data encoding format. There are two video decoding methods, hardware decoding, and software decoding. Software decoding generally uses the open source library FFMpeg. Audio decoding only supports software decoding; use FDK_AAC or OPUS decoding according to the audio encoding format. 6. Audio and video rendering Audio and video rendering is the last step in the video API processing process. This step seems to be very simple. You only need to call the system interface to render the data to the screen. However, one must process much logic to align the video picture with the audio. Still, now there is a standard processing method to ensure the synchronization of the audio and video. How to use ZEGOCLOUD video API Building a complete set of real-time audio and video systems is complex work. Nevertheless, many Video APIs help us solve the underlying complex operations. We only need to focus on the upper-level business logic. The following will introduce how to use the to implement the video calling function. ZEGOCLOUD Video API 1. Implementation process The following diagram shows the basic process of User A playing a stream published by User B: The following sections explain each step of this process in more detail. 2. Optional: Create the UI Before creating a instance, we recommend you add the following UI elements to implement basic real-time audio and video features: ZegoExpressEngine A view of the local preview A view for remote video An button End 3. Create a ZegoExpressEngine instance To create a singleton instance of the class, call the method with the of your project. ZegoExpressEngine createEngine AppID /** Define a ZegoExpressEngine object */
ZegoExpressEngine engine;

ZegoEngineProfile profile = new ZegoEngineProfile();
/** AppID format: 123456789L */
profile.appID = appID;
/** General scenario */
profile.scenario = ZegoScenario.GENERAL;
/** Set application object of App */
profile.application = getApplication();
/** Create a ZegoExpressEngine instance */
engine = ZegoExpressEngine.createEngine(profile, null); 4. Log in to a room To log in to a room, call the method. loginRoom /** create a user */
ZegoUser user = new ZegoUser("user1");


ZegoRoomConfig roomConfig = new ZegoRoomConfig();
/** Token is generated by the user's own server. For an easier and convenient debugging, you can get a temporary token from the ZEGOCLOUD Admin Console */
roomConfig.token = "xxxx";
/** onRoomUserUpdate callback can be received only by passing in a ZegoRoomConfig whose "isUserStatusNotify" parameter value is "true".*/
roomConfig.isUserStatusNotify = true;

/** log in to a room */
engine.loginRoom("room1", user, roomConfig, (int error, JSONObject extendedData)->{
    // (Optional callback) The result of logging in to the room. If you only pay attention to the login result, you can use this callback.
}); Then, to listen for and handle various events that may happen after logging in to a room, you can implement the corresponding event callback methods of the event handler as needed. engine.setEventHandler(new IZegoEventHandler() {

    /** Common event callbacks related to room users and streams. */

    /** Callback for updates on the current user's room connection status. */
    @Override
    public void onRoomStateUpdate(String roomID, ZegoRoomState state, int errorCode, JSONObject extendedData) {
        /** Implement the callback handling logic as needed. */
    }

    /** Callback for updates on the status of other users in the room. */
    @Override
    public void onRoomUserUpdate(String roomID, ZegoUpdateType updateType, ArrayList<ZegoUser> userList) {
        /** Implement the callback handling logic as needed. */
    }

    /** Callback for updates on the status of the streams in the room. */
    @Override
    public void onRoomStreamUpdate(String roomID, ZegoUpdateType updateType, ArrayList<ZegoStream> streamList, JSONObject extendedData){
        /** Implement the callback handling logic as needed. */
     }

}); 5. Start the local video preview To start the local video preview,  call the method with the view for rendering the local video passed to the parameter. startPreview canvas You can use a , , or to render the video. SurfaceView TextureView SurfaceTexture /**
 *  Set up a view for the local video preview and start the preview with SDK's default view mode (AspectFill).
 *  The following play_view is a SurfaceView, TextureView, or SurfaceTexture object on the UI.
 */
engine.startPreview(new ZegoCanvas(preview_view)); 6. Publish streams To start publishing a local audio or video stream to remote users, call the method with the corresponding Stream ID passed to the parameter. startPublishingStream streamID /** Start publishing a stream */
engine.startPublishingStream("stream1"); Then, to listen for and handle various events that may happen after stream publishing starts, you can implement the corresponding event callback methods of the event handler as needed. engine.setEventHandler(new IZegoEventHandler() {
    /** Common event callbacks related to stream publishing. */

    /** Callback for updates on stream publishing status.   */
    @Override
    public void onPublisherStateUpdate(String streamID, ZegoPublisherState state, int errorCode, JSONObject extendedData){
        /** Implement the callback handling logic as needed. */
    }
}); 7. Play streams To start playing remote audio or video stream, call the method with the corresponding Stream ID passed to the parameter and the view for rendering the video passed to the parameter. startPlayingStream streamID canvas You can use a , , or to render the video. SurfaceView TextureView SurfaceTexture /**
 *  Start playing a remote stream with the SDK's default view mode (AspectFill).
 *  The play_view below is a SurfaceView/TextureView/SurfaceTexture object on UI.
 */
engine.startPlayingStream("stream1", new ZegoCanvas(play_view)); 8. Stop publishing and playing streams To stop publishing local audio or video stream to remote users, call the method. stopPublishingStream /** Stop publishing a stream */
engine.stopPublishingStream(); If a local video preview is started, call the method to stop it as needed. stopPreview /** Stop local video preview */
engine.stopPreview(); To stop playing remote audio or video stream, call the method with the corresponding stream ID passed to the parameter. stopPlayingStream streamID /** Stop playing a stream*/
engine.stopPlayingStream(streamID); 9. Log out of a room To log out of a room, call the method with the corresponding room ID passed to the parameter. logoutRoom roomID /** Log out of a room */
engine.logoutRoom("room1"); Did You Know? 👏 Like and Follow is the biggest encouragement to me Follow me to learn more technical knowledge Thank you for reading :) Learn More This is one of the live technical articles. Welcome to other articles: All Live Streaming Features in One Article Build a Clubhouse Clone App with Android and ZEGOCLOUD How To Implement a Video Call App Using iOS CallKit and ZEGOCLOUD Also published . here

Flow

Google

Instagram

YouTube

How to Build a Live Streaming App

How To Implement a Video Call App Using iOS CallKit and ZEGOCLOUD

Learn More

Nominated for 2022 - Android Advocate of the Year

Nominated for 2022 - Ios Writer of the Year

Too Long; Didn't Read

Make resilience your competitive advantage

How to Implement a Simple Video Calling Feature Using the ZEGOCLOUD Video API

How to Implement a Simple Video Calling Feature Using the ZEGOCLOUD Video API

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Build a Clubhouse Clone App with Android and ZEGOCLOUD

Java bits: 0xFF and 0xFFL

10 Things You Didn't Know You Could Do With Google Keep

10 Things to check on Android code reviews

10 Mistakes In Mobile App Development That Make You Look Dumb

Build a Clubhouse Clone App with Android and ZEGOCLOUD

Java bits: 0xFF and 0xFFL

10 Things You Didn't Know You Could Do With Google Keep

10 Things to check on Android code reviews

10 Mistakes In Mobile App Development That Make You Look Dumb

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps