Data Production Management Crash Tips (10 minutes)

Comment 1 this document, you will learn how to use the MooreData Platform to complete the management of data production from 0 to 1. Here we take the annotation task of image data as an example.

1. Philosophy of Platform Design

1.1 Team space

Before starting to learn how the platform operates,it is helpful to understand the platform design philosophy of MooreData Platform.

As shown in the figure below, "team" is the basic unit of ABAKA AI ecosystem. The team can serve as an "isolated island"; complete the data labeling requirements within the team.

1.2 Data production

Delivery is the keyword of data production.

In the optimal data production solution of ABAKA AI, we regard the batch as the smallest unit of data delivery. In a data task, each data batch follows the same process for data production.

Taking the following data production process as an example, we have created a 4-node process. Node 1 carries out the labeling process, and nodes 2, 3, and 4 are all review processes. Only one team participates in each node. We will assign node 2 to the supplier team to complete the self-inspection, and node 3 will be assigned to our self-built quality inspection team to complete. Finally in node 4 , we will let the customer do the final quality inspection.

For example, this process is like a reimbursement process in the company. The first node is for you to submit the reimbursement application. The second node is for your superior to review, if it is not passed, it will be called back to you for modification, if it is passed, it will enter the third node. The third node is for the review by the manager, if it is not passed, it will be called back to you for modification. If it is passed, it will enter the fourth node. The fourth node is the final node. If it is passed, your reimbursement will come down. If it is not passed, it will be called back and start the process again.

You can choose a process template for data production according to your actual needs, such as 3 nodes (corresponding to double audits) or 5 nodes (corresponding to four audits).

nodes (corresponding to double audits) or 5 nodes (corresponding to four audits).Intuitively view the historical operations of the batch and the accuracy of each node.

After each node ends, the system will automatically count and analyze the behavior data on the node. As the manager of this data task, we can visually see the sampling accuracy of each node. In addition, we can also know many secrets of the data batch by viewing the details, which is of great significance for subsequent data production. For example, in the batch report, we can know which labeler has the lowest labeling accuracy, then by canceling the labeler's permission to continue annotation, the overall quality of the data can be improved.

1.3 Data concept

1.3.1 Task

Task, representing all the data under this task, will follow the same annotation rules, labelling setting, attribute setting, workflow, etc. for data production.

1.3.2 Batch

Batch, is the result of splitting a complete task, and is also the minimum unit of productivity process.&##x20;

1.3.3 Item

The item is each corresponding image/point cloud/text/video/questionnaire, etc. in the batch, and is also the smallest data unit that a labeler can get.

1.3.4 Frame

Frame is a concept that appears in continuous frame tasks, where an item can contain n frames of images/point clouds.

2. Team Setup

2.1 Create team

You can register for a MooreData platform account via theMooreData Registration Link. After successful registration, the platform will automatically create a team space for you called “username_team1”.

Click the button [Create Team] in the uper left corner to complete the creation of the team.

2.2 Add team member

To add members to your team, you need to follow these steps:
1. Go to [Team Manage].
2. Click on [Team Members].
3. Click on [Add Member].
4. Enter the personal ID of the new member you wish to add to your team.

To copy a personal ID, you can follow these steps:
1. Click the profile icon at the bottom left corner.
2. Choose [Personal Info].
3. Click on the “copy” button next to the Personal ID.

2.3 Setup administrator

As the creator of the team, you may not have much time to manage the team. You can set a reliable member of your team as an administrator to assist you in managing the team and tasks.

3. Create task

Click [Data Gallery] in the left toolbar;
Click [Create Task] in the upper right corner.

Integer Intelligent Platform provides2D image annotation tool 、3D point cloud object annotation tool、4D point cloud object annotation tool、phoneme tool、audio tool、questionnaire tool、 video multimodal tool、text annotation tool, you can select the corresponding tool according to the labeling requirements.

For the image data annotation task, we can select 2D Image Annotation Tool, enter the task name in the upper left corner, and then click [Next Step].

3.2 Select task-flow

We can select the number of review nodes as needed in the task process configuration.

For example, if you set the task flow of [Three Reviews], the task flow of Three Reviews includes 4 nodes: labeling node, review node 1, review node 2, and review node 3, which can correspond to labeling, supplier internal audit, ABAVA Platform internal audit respectively and Party A acceptance.

After selecting the task flow, click [Create], and we have successfully created a task~

3.3 Task settings

click [Task Setting],
Switch to [Labels Setting] at the top to configure the label type and label attribute settings.&##x20;
Click [Create] under the basic settings.

Attention：If the label in the label setting is not configured, it cannot be labeled after entering the labeling interface.

3.4 Import data

Click [Import Export] in the left toolbar,
Click [Create Import] on the [Import] page,
Select the data import method according to the prompts.

After clicking the import button, the page will automatically redirect to the import interface, where the progress of data import can be viewed in real time.

For more import operations, you can refer to our document “Data Import”. In consideration of professional users who will pre-annotate the data themselves, we have added support for importing data with pre-annotation results.

4. Batch Management

Click on [Data Batch] on the left toolbar to enter the batch list. You can select [Launch Batch] to officially put the data of the batch into the production process, or click [Import Data] to continue importing data.

If the batch is launched, importing data to the batch will no longer be possible.

5.1 Task Member Setting

Select [Member] on the left under this task,
Click on [Task Member],
Set the member as the administrator of this task in the rightmost part of the member list.
Click on [Task Team] to invite an entire team to be a member of this task.

5.2 Node Member Setting

Enter the batch
1. Click on the node button,
2. select members from the team in the pop-up window,
3. set members as the administrator or ordinary reviewer of this node.

Administrators can return the batch data to the labeling node, or pass it to the next review node. Members do not have this permission.
- If you are the task owner, you can set the team for any node;&##x20;
- If you are not the task owner, you can only set the corresponding members for the nodes assigned to you. For example,&##x20;
- If your team is at the review 2 node, then you can only operate the member setting of the review 2 node. Other nodes are not authorized to operate.
You can only set the members of your team when the data is pushed to the node where your team is located.

6. Quality Review

6.1 Claim Review

When a batch of data is pushed to your review node, you can click the [Start] button in the lower left corner to begin the review of that batch.

If you click on [Review Label] in the upper right corner without selecting any item, you will enter a random review mode, where items will be selected randomly for review.

If you select items and then click [Review Label], you will review the items you have chosen in sequence.

6.2 Items Awaiting Acceptance

For example, a member of Audit 2 node can see the items of Audit 3 node with error tags, but not the items of Audit 2 node with error tags.&##x20;

The "items to be Accepted" list displays the items that need to be modified by the current annotator, which are the items that had an error tag in the previous node.

7. Review Tools（2/3D Fusion Annotation)

7.1 Label Review

When reviewing an item, the review lock is locked by default to prevent accidental modifications to the data by the reviewers. If you want to modify the data content, you need to unlock the review lock, at this time, it will switch from review mode to edit mode.
In review mode, right-clicking on a label can pop up a label details page, which includes label type, label attributes, preset error reasons, and custom error reasons.

Checking the corresponding error items in the corresponding positions on the label details page, selecting the tag errors preset in the task settings, and choosing the custom error reasons, will all activate the error judgment button. Clicking on the judgment error can save the error review of the label.

All error labels will be displayed in the [Errors] on the right side, clicking on them will jump to the corresponding frame.

7.2 Omission

Click on the [Omission] icon in the upper left corner to enter the missed labeling mode.
Click on the missed labeling position in the 3D point cloud/2D image to pop up the omission window.
Choose the type of label missed or the description of the missed label,
click confirm, and the omission mark will be kept.

All missed labels will be displayed in the “Issues” on the right side.

7.3 Item Operation

Reviewers can perform operations such as save, skip, reject, or pass on the item.
- [Save]：Saving the current review or modification content, exiting the item or closing the webpage will not affect the data changes that have already occurred.
- [Skip]：If the item is skipped, the data changes that have already occurred will be retained, but the review result will not be included in the statistical analysis.
- [Reject]：Mark the item as an erroneous item.
- [Pass]：Mark the item as the correct item.

7.4 Batch Operation

After reviewing all items in this batch, if you are dissatisfied with the batch data, you can click on “Reject” in the lower left corner to reject the batch back to the labeling node for rework;
if the batch data is qualified, you can click on “Pass” in the lower left corner to push the batch to the next node.

7.5 Rework Review

In the list of items, you can drop down to filter the current [Passed] and [Unreviewed] items.

In the list of items, you can view the [Awaiting Acceptance] items, which are data that was found to be incorrect in the last review and have been corrected.