Engineering scene visual data simulation and door crane long tail sample generation case

Project background

This project originated from the visual data simulation needs of engineering equipment and infrastructure. The goal is to organize limited on-site samples, business scenario descriptions and target category requirements into data generation samples that can be displayed, reviewed, and continuously expanded. The project covers three types of scenarios: automatic driving of tunnel construction engineering vehicles, intelligent gantry cranes at railway ports, and intelligent detection of bridge structural diseases. Public cases only retain technical routes, deliverables and statistics, and do not disclose customer names, original site pictures, complete business documents, quotation details or internal communication information.

The initial project information includes door crane-related image data packages, engineering scene requirements documents, customer display materials and data collection requirements. Since only images and category tables were found in the original door crane data package, no detection frames or segmentation annotations were found. This round of delivery did not describe the results as a complete detection data set that can be directly trained, but was clearly positioned as a basic version for image-level simulation enhancement, synthetic sample generation, customer communication materials, and subsequent annotation planning.

Requirements dismantling and scenario design

When dismantling the requirements, the project divided the data generation task into three layers: the first layer is image enhancement of the gantry crane yard, which is used to simulate environmental changes that are common in real collection but difficult to cover at one time; the second layer is the generation of four types of target samples, used to complete the visual styles of common targets such as gantry cranes, containers, cone barrels, and forklifts; the third layer is engineering scenario cases for customer communication, covering three types of businesses: tunnels, ports, and bridges, so that demanders can intuitively see which targets, environments, and boundaries will be involved in subsequent collection, labeling, training, and evaluation.

Simulation types for the door crane scene include clear_day, night_yard, rain_camera, fog_haze, sunset_backlight, motion_blur, foreground_occlusion, and surveillance_compression. Each category generates 6 images, a total of 48 images, unified into 1280×720 JPG, and retains the metadata.csv record source, simulation scene, random seed and transformation parameters. The QA file shows that 48 pieces of metadata correspond to 48 images one-to-one, and no errors, unknown sources, external sources, low-information samples, or near-duplicate samples are recorded.

Data generation results

This project resulted in multiple sets of deliverables. The door crane simulation demo contains 48 image enhancement samples, simulation effect puzzle, original sampling puzzle, category table, metadata and quality check summary. The quality summary shows that the four original category prompt distributions are forklift 16, cone 15, person 13, and bollard 4; the brightness statistical range is 39.4, 94.55, 137.5, and the standard deviation statistical range is 15.5, 37.3, 62.6. This statistic is used to describe the visual distribution of samples in this round and is not used as a model performance indicator.

The four-category target synthetic data set includes four categories: crane, container, cone, and forklift, with 10 images in each category, a total of 40 images, and parameters such as background, rotation angle, shearing, scaling, horizontal flip, target center, and landing point are recorded. There is also an AI synthetic sample set that also covers the four categories of forklift, crane, container, and cone. There are 10 pictures in each category, a total of 40 pictures, for customers to quickly check the visual performance of different target categories in engineering scenarios. The project also generated 3 examples of foreground pose and background fusion to illustrate possible paths for angle changes and background replacement.

Eight engineering scene case materials

The project organized an additional 8 visual customer case materials, and provided supporting pictures, lightweight OBJ three-dimensional scenes, JSON metadata, CSV index and overview diagram for each case. Cases include: tunnel construction vehicle meeting and obstacle identification, port night rain and fog door crane grab box and keyhole identification, tunnel entrance and exit drastic changes in light and shade and drivable area identification, port daytime box number

The function of these materials is to transform abstract data synthesis requirements into discussable business objects, difficult working conditions and annotation ranges. For example, the port crane case with rain and fog at night includes door cranes, spreaders, containers, trucks, keyholes, locks, and box lines; the drone case under the bridge includes cracks, peeling, exposed ribs, rust, watermarks, and complex texture negative samples. The public page shows simulation examples and does not use the original images of the customer's site.

Implementation process

The project process is divided into data inventory, scenario modeling, sample generation, metadata sorting, quality inspection and customer material output. Data inventory is used to confirm which categories the original data contains, whether it is labeled, and whether it can be directly used for model training. Scenario modeling is used to break down port gantry cranes, tunnel construction, and bridge inspections into target categories, environmental variables, collection perspectives, and annotation types. The sample generation stage constructs changes such as nighttime, rain and fog, backlighting, motion blur, foreground occlusion, low visibility, compressed image quality, and surface stains according to long-tail working conditions.

The metadata organization stage records the source, scene, random seed, enhancement parameters, category prompts and output path for each sample to facilitate subsequent tracing. The QA phase focuses on sample number, file existence, source consistency, low-information images, near-duplicate images, brightness and texture differences. The customer material stage outputs pictures and 3D case packages, data collection requirements, customer display cases and collection instructions, so that the demander can clarify the videos, sample pictures, target close-ups, difficult working conditions, meta-information, calibration and 3D data that need to be supplemented before the project is initiated.

Applicable value

This type of visual data simulation service is suitable for visual algorithm projects with high on-site collection costs, few long-tail working conditions, many privacy or security restrictions, and lack of complete annotation in the early stages of the project. It can help the team unify the requirements before actual collection, expose target categories, annotation granularity, and working condition coverage gaps in advance. It can also be used as display material, collection list, and data closed-loop starting point in the solution validation stage.

For gantry cranes and port yard projects, data simulation can help evaluate conditions such as nighttime rain and fog, sand and dust obstruction, container contamination, vehicle parking, spreader motion blur, small target keyholes and container line obstructions. For tunnel and bridge projects, data simulation can help sort out issues such as low illumination, water mist, mud, strong backlight, complex structural textures, targets with small cracks, and changes in drone perspective.

delivery boundary

This case does not directly equate the simulation sample with the completion of model training, nor does it write the image-level enhancement results as a complete annotated data set with detection boxes or segmentation masks. If you subsequently enter YOLO, Detectron, segmentation model, OCR model or 3D reconstruction model training, you need to supplement the target frame, segmentation area, OCR character annotation, camera parameters, 3D calibration, real negative samples and on-site verification collection. Simulation data should be mixed with real data for verification, and a reproducible training, evaluation and regression process should be formed based on the target site.

FAQ