On-Demand Live Stream Playback with Amazon IVS (3 Part Series) 1 Auto Recording Amazon IVS Live Streams to S3 2 How to Archive Web Chat Messages with Amazon IVS Chat Logging 3 Amazon IVS Live Stream Playback with Chat Replay In our , we looked at how to auto-record Amazon Interactive Video Service (Amazon IVS) live streams to an Amazon Simple Storage Service (Amazon S3) bucket. That feature is a powerful tool for user generated content (UGC) platforms as it is the first step in providing on-demand viewing of previous live streams in an application. But a replay of a live stream is incomplete without the full context of the interactive chats that occur alongside of the stream. In this post, we'll take the next step to providing a full on-demand experience by learning how to log Amazon IVS chat messages. last post Logging Configurations Similar to how auto-recording to Amazon S3 required a recording configuration, chat logging also requires a configuration that defines the necessary details for a chat room. Keep in mind that you might also need additional depending on your choice of logging destinations. IAM permissions Using the Amazon IVS Console To Create a Logging Configuration To create a logging configuration with the Amazon IVS Console, click on in the left sidebar. Logging Configurations On the list page, click . Logging configurations Create logging configuration Amazon IVS chat logging offers several destinations for storage of the logged chat messages. We can choose from Amazon CloudWatch, an Amazon Kinesis Data Firehose, or Amazon S3 as a storage destination for your chat logs. I personally find CloudWatch to be the most convenient destination since I can easily retrieve the chat messages via the CloudWatch SDK, so let's use that for this demo. Refer to the if you'd like to utilize an Amazon Kinesis Data Firehose or Amazon S3 as a destination in your application. documentation : There can be a slight delay between when a chat message is posted and when it appears at your logging destination. For Amazon S3, the delay can be up to 5 minutes, and for Amazon CloudWatch and Amazon Kinesis Data Firehose it can be up to 10 seconds. Note To create the configuration for this demo, enter a (#1), select CloudWatch as the (#2), select (#3), and enter a (#4). Logging configuration name Destination Create a new CloudWatch log group Log group name Enter any optional and then click . Tags Create logging configuration Using the AWS SDK To Create a Logging Configuration Just like with recording configurations, we can take advantage of the AWS SDK to create our logging configurations. Since we're logging to CloudWatch in this post, we'll need to use the CloudWatch Logs SDK to . create a log group import { CloudWatchLogsClient, CreateLogGroupCommand } from "@aws-sdk/client-cloudwatch-logs"; const cloudWatchLogsClient = new CloudWatchLogsClient(); const createLogGroupInput = { logGroupName: 'ivs-demo-chat-logging-group', }; const createLogGroupRequest = new CreateLogGroupCommand(createLogGroupInput); const createLogGroupResponse = await cloudWatchLogsClient.send(createLogGroupRequest); console.log(createLogGroupResponse); Now we can use the Amazon IVS chat client to create a logging configuration ( ). docs import { IvschatClient, CreateLoggingConfigurationCommand } from "@aws-sdk/client-ivschat"; const ivsChatClient = new IvschatClient(); const createLoggingConfigInput = { name: 'ivs-demo-chat-logging-config-sdk', destinationConfiguration: { cloudWatchLogs: { logGroupName: 'ivs-demo-chat-logging-group' } } }; const createLoggingConfigRequest = new CreateLoggingConfigurationCommand(createLoggingConfigInput); const createLoggingConfigResponse = await ivsChatClient.send(createLoggingConfigRequest); console.log(createLoggingConfigResponse); This will produce output similar to the following: { "$metadata": { "httpStatusCode": 200, "requestId": "[redacted]", "cfId": "[redacted]", "attempts": 1, "totalRetryDelay": 0 }, "arn": "arn:aws:ivschat:us-east-1:[redacted]:logging-configuration/[redacted]", "createTime": "2023-01-09T14:48:35.358Z", "destinationConfiguration": { "cloudWatchLogs": { "logGroupName": "ivs-demo-chat-logging-group" } }, "id": "[redacted]", "name": "ivs-demo-chat-logging-config-sdk", "state": "ACTIVE", "tags": {}, "updateTime": "2023-01-09T14:48:35.485Z" } Creating a Logging Configuration with the AWS CLI We can also optionally create a logging configuration via the AWS CLI. Again, we'll have to create the CloudWatch log group, then pass the name of the new log group into the chat logging configuration. $ aws logs \ create-log-group \ --log-group-name ivs-demo-logging-config-log-group $ aws ivschat \ create-logging-configuration \ --name ivs-demo-logging-config \ --destination-configuration cloudWatchLogs={logGroupName=ivs-demo-logging-config-log-group} Associating a Logging Configuration With an Existing Amazon IVS Chat Room via the Amazon IVS Console We can now associate the logging configuration with an existing Amazon IVS chat room. Select under in the left sidebar, choose the room that you'd like to associate the configuration with, and then click . Rooms Chat Edit On the room edit page, scroll down and select (#1). In the dropdown (#2), search for and associate the configuration that we created above. Note that you can associate multiple configurations with a room, so if you'd also like to log to an Amazon Kinesis Data Firehose or Amazon S3, you can create separate configurations and associate them with your room. Also note that you can create a brand new configuration directly from the edit room page by clicking on the button (#3). Automatically log messages and events Logging configurations Create logging configuration Associating a Logging Configuration With an Existing Amazon IVS Chat Room via the AWS SDK In production, you'll likely use the SDK to associate your logging configuration with a new or existing chat room. To do that, use ( ) or ( ). Here's an example of updating an existing chat room with a chat logging configuration. As expected, we'll need to pass the ARN as the of the chat room, and any/all chat logging configuration ARNs in an array to . CreateRoomCommand docs UpdateRoomCommand docs identifier loggingConfigurationIdentifiers import { IvschatClient, UpdateRoomCommand } from "@aws-sdk/client-ivschat"; const ivsChatClient = new IvschatClient(); const updateChatRoomInput = { identifier: 'arn:aws:ivschat:us-east-1:[redacted]:room/[redacted]', loggingConfigurationIdentifiers: [ 'arn:aws:ivschat:us-east-1:v:logging-configuration/[redacted]', ] }; const updateChatRoomRequest = new UpdateRoomCommand(updateChatRoomInput); const updateChatRoomResponse = await ivsChatClient.send(updateChatRoomRequest); console.log(updateChatRoomResponse); You can specify multiple logging configurations for a chat room. This lets you use multiple destinations, if desired. The array of chat logging identifiers that you pass via the SDK to will overwrite any existing identifiers, so if you are adding an additional configuration, make sure to include any existing configuration ARNs to avoid removing them. Note: UpdateRoomCommand Retrieving Chat Logs At this point, all new messages posted to an Amazon IVS chat room will be logged to the destination(s) specified by the attached logging configuration(s). As mentioned above, in the case of CloudWatch as a logging destionation, we can use the CloudWatch SDK to retrieve our chat logs for a given time period. To do this, we need the , the start and end timestamp (in Unix timestamp format), and the . The will be in the format: . So, for a chat room with an ARN of , the will be . logGroupName logStreamNames logStreamName aws/IVSChatLogs/1.0/room_[suffix of chat room ARN] arn:aws:ivschat:us-east-1:[redacted]:room/0wgOPVl4ZRdu logStreamName aws/IVSChatLogs/1.0/room_0wgOPVl4ZRdu Retrieving CloudWatch Logged Chat Messages Here's how to use the to construct a ( ) that is used to retrieve chat logs. CloudWatchLogsClient FilterLogEventsCommand docs import { CloudWatchLogsClient, FilterLogEventsCommand } from "@aws-sdk/client-cloudwatch-logs"; const cloudwatchClient = new CloudWatchLogsClient(); const filterLogEventsInput = { logGroupName: 'demo-chat-log-cw-group', logStreamNames: ['aws/IVSChatLogs/1.0/room_0wgOPVl4ZRdu'], startTime: 1672929210000, endTime: 1672929330000, }; const filterLogEventsRequest = new FilterLogEventsCommand(filterLogEventsInput); const filterLogEventsResponse = await cloudwatchClient.send(filterLogEventsRequest); const events = filterLogEventsResponse.events.map(e => JSON.parse(e.message)); Remember that when streaming to a channel. You might use a stream start and end time if you're trying to retrieve chat logs for a specific Amazon IVS live stream. How do I know which start and end time to use? Amazon IVS sends events to EventBridge Keep in mind, that the can not return an infinite number of events. You may have to check the for the existence of a and handle pagination if your time period has more events than can fit in a single response from the SDK. Refer to the SDK for more information. FilterLogEventsCommand FilterLogEventsCommandOutput nextToken docs By default, this operation returns as many log events as can fit in 1 MB (up to 10,000 log events) or all the events found within the specified time range. If the results include a token, that means there are more log events available. You can get additional results by specifying the token in a subsequent call. This operation can return empty results while there are more log events available through the token. At this point we can do whatever we need with the chat log. Remember, chat logging will log posted to an Amazon IVS chat room, including custom events and events related to chat moderation. If we wanted to replay a chat stream, we'll probably need to parse the events a bit to clean up moderated messages and construct a "replayable" stream of messages. all events : If you haven't read them yet, check out my blog posts on and chat moderation with Amazon IVS chat. More About Moderating Chat automated manual A function to parse the logged events might look similar to this. Your mileage may vary! const parseEvents = (events) => { let parsedEvents = []; events.forEach(e => { switch (e.type) { case 'MESSAGE': parsedEvents.push(e); break; case 'EVENT': if (e.payload.EventName === 'aws:DELETE_MESSAGE') { const existingEventIdx = parsedEvents.findIndex(parsedEvent => { return parsedEvent.payload.Id === e.payload.Attributes.MessageID }); if (existingEventIdx > -1) { parsedEvents.splice(existingEventIdx, 1); } } break; } }); return parsedEvents; }; This function will return an array that is suitable for chat replay purposes with moderated messages removed from the chat log. Here's a small example of how the event stream might look. [ { "event_timestamp": "2023-01-05T14:33:32.894Z", "type": "MESSAGE", "payload": { "Type": "MESSAGE", "Id": "WhO6MW6iRdS5", "RequestId": "", "Attributes": { "username": "gleningp" }, "Content": "bbiab!", "SendTime": "2023-01-05T14:33:32.894089757Z", "Sender": { "UserId": "75758272-f3f2-4f65-83c5-9b8f144116b8", "Attributes": {} } }, "version": "1.0" }, { "event_timestamp": "2023-01-05T14:33:39.896Z", "type": "MESSAGE", "payload": { "Type": "MESSAGE", "Id": "VwEPwPV74GN3", "RequestId": "", "Attributes": { "username": "rpeirazzia" }, "Content": "perfect", "SendTime": "2023-01-05T14:33:39.896519733Z", "Sender": { "UserId": "704D6BF8-22D2-4A52-B6A7-BDEFB115ECE5", "Attributes": {} } }, "version": "1.0" }, { "event_timestamp": "2023-01-05T14:33:47.330Z", "type": "MESSAGE", "payload": { "Type": "MESSAGE", "Id": "00kqu1sPa6dF", "RequestId": "", "Attributes": { "username": "jmycroft2" }, "Content": "🔥🔥🔥", "SendTime": "2023-01-05T14:33:47.330983449Z", "Sender": { "UserId": "2F12AA2D-DF65-42CF-AE99-133A5A06F7B4", "Attributes": {} } }, "version": "1.0" } ] Summary In this post, we learned how to log messages posted to an Amazon IVS chat room. In our next post, we'll look at bringing together auto-recorded live streams and logged chat messages in an application to create a suitable "on-demand" replay user experience.
