Overview

Nonolive is a China’s overseas mobile video live broadcast platform. It was first launched in Indonesia in early June 2016. Its products cover iOS and Android.

At present, there are hundreds of anchors on the Nonolive live broadcast platform to provide overseas users with singing,
Rich pan-entertainment content such as dances and challenges is part of Guangzhou Shixun Information Technology Co., Ltd.

Currently, the Nonolive live broadcast platform is deployed in the Singapore region of cloud vendors. The main resources used are EC2, Aurora, and ALB. As business development demands, Nonolive’s live broadcast platform needs to report live broadcast statistics for back-end managers. This report It needs to be calculated and analyzed based on a large amount of live data, and it is required to add a data processing process on the basis of existing resources.

In response to this demand, Agile Cloud needs to assist customers in determining the data processing process plan and deploying it in the production environment.

The customer’s previous architecture

Currently, the Nonolive live broadcast platform is deployed in the Singapore region of cloud vendors. The main resources used are EC2, Aurora, and ALB. The current structure is as follows:

Requirement

Currently, the Nonolive live broadcast platform is deployed in the Singapore region of cloud vendors. The main resources used are EC2, Aurora, and ALB. With the needs of business development, Nonolive’s live broadcast platform needs to report live broadcast statistics for back-end managers. This report It needs to be calculated and analyzed based on a large amount of live data, and it is required to add the data processing process to the existing resources. The existing resources of the customer

According to the current situation of customers, it is necessary to add the existing data analysis and processing architecture. The customer’s current data storage is on Aurora and Ec2, where the location of the data storage needs to be unified. Considering the tight time for the customer’s business to go online, and for the convenience of the customer’s later maintenance work, the data analysis cluster of the Elastic MapReduce(EMR) cluster is selected, and the Elastic MapReduce(EMR) cluster can be combined It is convenient to load data from S3, so the data needs to be synchronized to the S3 bucket in a unified manner.

Because the data in Aurora is table data, it needs to be exported as a file and stored in s3 in real time. The DMS migration service on cloud vendors can facilitate data synchronization. Use DMS to synchronize aurora data to S3 in real time.

The second is the configuration of the customer’s Elastic MapReduce(EMR) cluster. The customer is not familiar with the data analysis service on the cloud, and needs to assist the customer to complete the Elastic MapReduce(EMR) cluster deployment in the test and production environment, and adjust its functions.

Architecture description

Architecture description

1. 1. Configure the front-end application to save the data that does not need to be stored in the database but needs to be analyzed in the S3 data bucket
2. 2. Use the DMS data migration service to synchronize Aurora data to the S3 data bucket
3. 3. Configure the Elastic MapReduce(EMR) service to load the file data in S3
4. 4. Set the Elastic MapReduce(EMR) service to run data tasks and save the data structure to another S3 result bucket
5. 5. Configure the front-end application to obtain data report information from the S3 result bucket

Effect

Agilewing helps customers complete the design, construction and key process testing of the entire platform to achieve rapid platform deployment and launch. At the same time, using Elastic MapReduce(EMR) service to reduce the work pressure of the operation and maintenance team and the cost of the data analysis platform