AprilTag Fiducials

AprilTags are a special type of QR Code like marker that are use for computer vision recognition. They feature a unique black and white pattern that can be easily detected and uniquely identified. On FRC fields, they are used as a reference point for robots to determine their position on the field relative to the field elements. AprilTags are often placed on or near the most important field elements, such as scoring areas or game piece pickup locations.

AprilTag Detection

To detect AprilTags, we use the algorithm described in the WPILIB documentation for AprilTag detection. Unlike the reference implementation, which uses the AprilTag C library, we use a GPU accelerated version of the library, originally written by FRC Team 971. This version of the library is optimized to run on Nvidia Jetson vision computers, allowing for real time detection of AprilTags from multiple 60 FPS camera streams.

Pose Estimation

Once an AprilTag is detected in an image, we can then use the known size of the AprilTag to estimate a camera relative pose (position and orientation) of the AprilTag. This is done using a mathematical technique called Perspective-n-Point (PnP). The PnP algorithm takes as input the 2D coordinates of the corners of the AprilTag in the image, the known 3D coordinates of the corners of the AprilTag in the real world, and the camera's calibration parameters (camera intrinsics). Using this information, the PnP algorithm computes the 3D position and orientation of the AprilTag relative to the camera.

For the purposes of localizing the robot, we can also compute the pose of the camera in field space by providing the 3d positions of the tag corners in field space, rather than relative to the camera, and using multiple tags in the field of view, we can get a more accurate estimate of the camera's pose.