Skip to main content

Case 11: Gesture-Controlled Excavator

Case Introduction

Build a gesture-controlled excavator and control the excavator's movement and actions through gestures.

Teaching Preparation

NameIllustration
Nezha Pro AI Mechanical Power Kit

Teaching Objectives

  1. Complete the assembly of the gesture-controlled excavator, master the assembly and connection methods of the excavator chassis, excavator arm (including transmission components), and gesture recognition sensor, and realize the control of the excavator's movement, excavator arm lifting, and bucket opening/closing through gestures.
  2. Understand the principle of "multi-action collaboration" of the excavator (e.g., movement + excavator arm action), recognize the correspondence between gesture commands and excavation actions, and learn to program to coordinate the actions of different components.
  3. During the process of debugging the excavator arm's movement range and bucket opening/closing angle, cultivate hands-on ability, logical thinking, and spatial imagination, and learn to solve problems such as stuck excavation actions and the bucket's inability to scoop up "simulated soil" (e.g., small stones).
  4. Stimulate interest in engineering robots, perceive the technical charm of "intelligent excavation", and establish the cognition that "technology supports engineering construction".

Story Introduction

While organizing the warehouse, everyone found that the stock of energy crystals was almost exhausted. "The crystals are hidden in the mine to the east," Aji brought up the map. "But the excavator 'Zuandishu (Ground-Drilling Rat)' is on strike—it needs gesture commands to direct the excavation direction." At the mine entrance, the children saw that Zuandishu's drill bit was stuck in the rock. "When you wave your hand to the left, it excavates to the left; when you wave downward, it digs deeper," Aji demonstrated. "It’s just like a conductor directing a symphony—each gesture has a special meaning."

Learning Exploration

  1. Which components (e.g., motor, servo) control the excavator arm and bucket of the excavator respectively? Try using gesture commands (e.g., waving forward = excavator arm descending, waving backward = excavator arm ascending, making a fist = bucket closing) to control excavation actions. How to ensure smooth actions?
  2. Will the excavator experience insufficient power when controlling the excavator arm while moving? How to optimize this through programming (e.g., prioritizing movement power before controlling the excavator arm)?
  3. If the bucket cannot close tightly (causing "simulated soil" to leak out), what might be the causes? (e.g., loose connection of bucket components, insufficient servo rotation angle)
  4. Compare the structures of real excavators and the model—how do real excavators' arms achieve "high-force excavation"? (Hint: Refer to the hydraulic system) Which function of real excavators do the model's transmission components simulate?

Assembly Steps

Hardware Connection

Connect the gesture recognition sensor to the IIC interface of the Nezha Pro Expansion Board, and connect the three smart motors to the M1, M2, and M3 interfaces of the Nezha Pro Expansion Board respectively.

Code Programming

Go to "makecode.microbit.org" and click New Project.

Enter the project name in the pop-up window and click Create.

Click Extensions in the code drawer.

In the pop-up interface, enter nezha pro and click the search icon. After the nezha pro software library appears, click it. Load the PlanetX software library in the same way.

Sample Program

Program link: https://makecode.microbit.org/_ik31qLKCJW5C

You can also download the program directly through the following webpage.

Program Download

Use a USB cable to connect the PC and micro:bit V2.

After successful connection, a drive named MICROBIT will be recognized on the computer.

Click the icon at the bottom left and select Connect Device.

Click .

Click .

In the pop-up window, select BBC micro:bit CMSIS-DAP, then select Connect. Now, the micro:bit has been successfully connected.

Click Download Program

Case Demonstration

After turning on the power, control the excavator's movement and actions through gestures.

Extended Knowledge

  1. Core components of real excavators: Real excavators rely on hydraulic systems to provide powerful power. The lifting of the excavator arm and the opening/closing of the bucket are driven by hydraulic cylinders, which can scoop up tons of soil or rocks. This case uses motors/servos to simulate the "action control" function of the hydraulic system.
  2. Development of intelligent excavators: Modern excavators have begun to integrate GPS positioning, visual recognition, and gesture control technologies—for example, using gestures to control the precise positioning of the excavator arm, or using AI algorithms to identify excavation objects (e.g., distinguishing between soil and rocks) to improve excavation efficiency.
  3. Application scenarios of excavators: Excavators are core equipment in engineering construction, used in house construction (foundation excavation), road construction (earthwork), mine mining (ore excavation), and other fields. The progress of their mechanical structure and control technology directly affects project progress.