Team: Team 07, "The Console"

Members:

• Arnav Thapar (arnavthapar)
• Pranav Emmadi (CyberBrainiac1)
• Lucas Escobar (NotXander4211)

Summary: SolderBuddy is an AI-powered collaborative robot built on an AMD AI PC and SO101 robot arms. The system assists users during soldering by autonomously retrieving and handing over tools, allowing the human to focus on precision electronics work while the robot manages tool delivery..

• Real world application of the mission
  SolderBuddy improves electronics assembly by acting as a robotic assistant. While soldering, it gets and gives you tools such as solder wire, solder suckers, or wire strippers on demand, reducing interruptions and increasing efficiency for users working on electronics projects..

• How this can be used outside of soldering
  Instead of using our system for soldering, our project could be adapted into many other sectors. For example, the project can be adapted for use in the medial sector and operating rooms. It could be used to had over tools or similar to a sugeon, creating the opportunity for less people on a single sugery allowing more surgeries to happen at the same time!!

• The Approach
  Instead of trying to fully automate soldering, SolderBuddy focuses on human-robot collaboration. It does this by supporting the user by handling tools while keeping precise control with the human..
• Innovation in design and application
  The system integrates AI vision, robotic manipulation, and a physical macro pad interface to create an intuitive and low-distraction workflow during soldering tasks. In the future, we want to integrate a GUI where the user can quickly glance to see which tools they have and/or check on their robot..

• Multiple cameras are used to visualize the workspace and detect tool locations.
• Visual data is captured from the workbench to train AI models for object recognition and pickup.

• AI models are trained to recognize soldering tools and determine grasp locations.
• Training and experimentation are performed on an AMD AI PC.
• We went through approximately 480 episodes, this was due to switching models, combining datasets, changing number of cameras, adding barrier, changing placement of mat and other things on the view, changing view, and data not saving.

• During runtime, the system performs real-time inference to locate the requested tool.
• The SO101 robot arm autonomously picks up the tool and presents it to the user.
• The arm waits for the tool to be removed before returning to its home position.

• Generalizability of implementation across tasks or environments The perception and pickup pipeline can be retrained for additional tools or different workbench layouts..
• Flexibility and adaptability of the solution Tool selection is mapped to macro pad buttons, allowing easy reassignment without changing core robot logic..
• Types of commands or interfaces needed to control the robot A physical macro pad provides simple button-based commands for requesting tools..

• Value of mentorship
  Even when you think you can solve a problem on your own, having knowledgeable mentors makes a huge difference. Accepting their guidance helps you solve challenges faster and learn more effectively. (Thanks a lot, AMD!))
• Power of collaboration
  Brainstorming with teammates uncovers blind spots and sparks creative solutions by combining different perspectives and expertise. (Thanks, Alex!))
• Impact of hackathons
  The intensity and collaboration fuel curiosity and a passion for deeper learning in robotics and AI, leaving participants energized and motivated to continue exploring..
• Diversity of skill sets
  Teammates with different expertise (hardware, software, AI) allow challenges to be tackled from multiple angles simultaneously, increasing problem-solving potential..
• Importance of community
  Sharing knowledge with other teams and receiving guidance from mentors enriches the experience far more than simply competing or winning..

1. Power and fully set up both SO101 robot arms.
2. Connect all three cameras to the AMD AI PC.
3. Connect both robot arms to the PC via USB-C.
4. Edit .env to contain your correct ports, then run main.py
5. Press a button on the macro pad to request a tool.
6. The robot retrieves the tool, presents it to the user, waits for removal, and then returns to its origin position.

1. Bad train data -- slow, steady, and smooth recordings
2. Bad camera placement -- add a 4th camera
3. The ACT model not being able to give us a good output -- tried the pi0.5 and smolvl model
4. Claw not being able to get a good grip (especially on the spool and the pliers) -- approached them differently after many episodes of testing
5. Some webcams weren't connecting -- found out that each usb port only has the bandwith to support two webcams (split the webcams up)
6. cameras moved -- rerecord all the episodes

• https://huggingface.co/datasets/Cyberbrainiac/toolsmasterss
• https://huggingface.co/datasets/Cyberbrainiac/toolsmasterwire
• https://huggingface.co/datasets/Cyberbrainiac/toolsmastersolder

• https://huggingface.co/Cyberbrainiac/act_ss_16ksteps
• https://huggingface.co/Cyberbrainiac/act_wire_16ksteps
• https://huggingface.co/Cyberbrainiac/act_solder_16ksteps

• Leader SO101 Robot Arm
• Follower SO101 Robot Arm
• AMD AI PC
• AMD Flexible Camera
• UGreen 1080p Webcam
• Small camera mounted on claw
• Ring Light

• Soldering wire
• Soldering sucker
• Wire strippers

• Seeed XIAO RP2040
• Through-hole 1N4148 diodes
• 6× MX-style switches
• 0.91" OLED display
• EC11 rotary encoder
• 6× blank DSA keycaps
• 3× heat-set inserts

AMD_Robotics_Hackathon_2025_SolderBuddy/
├── README.md
├── TRAINING.md
├── LICENSE
└── mission
    ├── code
    │   ├── main.py
    │   └── .env
    └── wandb
        └── runs
─ runs