SoftServe and CTU-UPENN-NYU Team #1 in MBZIRC 2020 Robotics Challenge

The Mohamed Bin Zayed International Robotics Challenge (MBZIRC) is a biennial international robotics competition organized by Khalifa University.  This year’s competition took place at the Abu Dhabi National Exhibition Center, featuring 32 teams representing 19 countries, shortlisted from a total of 134 teams from around the globe.

SoftServe and our CTU-UPENN-NYU team (Czech Technical University in Prague, University of Pennsylvania, and New York University) earned top scores across multiple categories — most notably in the event’s Grand Challenge Triathlon, where they secured the #1 overall position.

The total prize for the competition was $250,000 for each challenge and $500,000 for the Grand Challenge.

MBZIRC consists of a series of challenges aimed at pushing technological and application boundaries in robotics, including:

• Fast autonomous navigation in complex unstructured environments
• Low visibility environments
• Tracking dynamic objects in 3D
• Sensing and avoiding obstacles
• RTK GPS-denied navigation in indoor-outdoor environments
• Physical interactions

Participating teams used autonomous aerial and ground robots to perform a range of tasks during the competition. While some teams received penalties for manual robot operation or oversized drone usage, teams with strong expertise in autonomous control had the biggest advantage in the challenge.

A few teams — CTU-UPENN-NYU and the SoftServe team in particular — had fully autonomous robots, including both UAVs and UGVs.

In addition to placing first overall, the CTU-UPENN-NYU and SoftServe team also won:

1. First place for “Team of Robots to Construct a Structure Using Bricks”
2. Second place for “Team of UAVs (Unmanned Aerial Vehicle) to Capture Flying Objects”

Our team utilized real-time computer vision algorithms to enhance thermal cameras for firefighting. Moreover, we incorporated artificial neural networks to track multiple objects following complex 3D trajectories. Additionally, the Robot Operating System (ROS) ensured that all solution modules communicated reliably, and sensor data processing was enhanced by applying optimal algorithms, ensuring that each challenge was completed successfully.

When preparing for the challenges, our team implemented many candidate solutions, with final testing conducted just days before the event. This testing revealed which implementations for particular tasks needed to be included in the final software and hardware solution to win the competition.

As a result, for the object detection challenge (Challenge 1), the decision was made to use the 3D LIDAR Oyster (high-resolution LIDAR sensors for long-, mid-, and short-range applications) in combination with neural networks. LIDAR was used to detect objects (a ball that had to be intercepted), while a neural network validated the results. This approach gave our team the advantage of detecting objects much further away compared to using only neural networks and classic computer vision techniques.

Additionally, SoftServe’s team implemented an approach for fire center detection that allowed for more precise fire suppression, earning a higher score in the fire challenge.

USE CASES

The showcased challenges are not only about technological and scientific development —they are also about boosting real-life industrial use cases.

Let’s take the Notre-Dame de Paris architectural tragedy as an example. Not only was it difficult for firefighters to douse the flames on the roof, but it was also challenging to adequately estimate the fire’s growth stages in real time. Fire-proof drones equipped with thermal cameras can detect fire spreading across a building, expedite the search and rescue of people trapped inside, and maximize the effectiveness of real-time situation tracking. Drones can also support quick post-fire and disaster assessments.

In the case of territory-wide open-air fires, such as the recent bushfires in Australia, drones can analyze vast areas in a matter of hours rather than days, enabling more accurate and faster decision-making.

Another significant real-world issue is managing safety around airports. Drones can efficiently patrol airports and similar restricted open-air facilities to effectively detect and prevent drone and human intrusions.

To address these critical challenges, SoftServe and other participants of MBZIRC have dedicated an enormous amount of time and involved the best engineering minds to ensure the devised solutions are highly effective, error-free, and ready for deployment in any critical situation.

Congratulations to all finalists on an amazing competition! We look forward to showcasing the continued robotics advancements of SoftServe’s and the CTU-UPENN-NYU team in 2022.