Real-time Face Recognition with YOLO

This device integrates real-time face detection with a 5-point facial landmarking system to identify known and unknown faces. It serves multiple applications, from a face recognition-based doorbell to an attendance and time management system for MSMEs with fewer than 100 employees.

Table 1 details the required components, while Fig. 1 shows the author’s prototype.

We previously designed a smart AI robot that can recognize a person’s face. If you are interested, you can check that too.

– Advertisement –

MaixCam Dev Board

The MaixCam is popular for edge AI applications and prototyping new AI algorithms. It is well-supported with libraries like MaixPy, a Python-based framework for K210 boards, making it accessible to both beginners and experts.

It uses GPIO pins to control external devices, such as a gate mechanism for employee access. LEDs are used as visual indicators. The GPIO pin map for the MaixCam board is used to control peripherals like LEDs or servo motors for the gate mechanism. A green LED lights up for recognised (known) faces, while a red LED lights up for unrecognised (unknown) faces.

– Advertisement –

Five-point Face Recognition System

A five-point face recognition system is used here. The system focuses on identifying five critical landmarks on a face. These five points are the two eyes, the nose tip, and the two corners of the mouth. These landmarks are essential for alignment and recognition in most face detection algorithms.

YOLOv8 is the latest version of the YOLO (You Only Look Once) family. It is faster, more accurate, and more efficient. Face detection locates faces within an image, while landmark detection pinpoints five key facial points. Recognition then matches the detected face with a stored database.

Operating with YOLOv8 offers several advantages, including high accuracy, ensuring precise detection even in low-light or cluttered environments. It is highly efficient, running in real time on embedded systems. Additionally, its versatility allows it to detect objects, landmarks, and more within a single model.

Hardware

For AI processing, the Sipeed microcontroller is used, featuring the following specifications: a dual-core CPU with a Big Core running at 1GHz (RISC-V C906/ARM A53) and a small core at 700MHz (RISC-V C906/ARM A53).

It includes an NPU delivering 1 TOPS@INT8 (1 TOPS =1 trillion 10¹²) operations per second), 256MB DDR3 RAM, and a TFT touchscreen.

The system runs a Linux-based operating system specifically designed for Kendryte K210 boards. Key features include integrated support for YOLOv8, a headless mode for efficient remote operations, Python 3.11 for scripting and automation, and an SSH daemon for secure remote access.

This setup is ideal for real-time face recognition, making it a great choice for entry-level AI models or lightweight edge AI systems.

Connecting and Setting up the Board

To begin, download the latest MaixCam operating system from the release page (https://github.com/sipeed/MaixPy/releases).

However, in most cases, the board will already have an operating system installed.

If so, only a connection to the computer is required. There are three primary methods to connect the board:

1. USB direct connection
2. Through SSH
3. Using MaixVision IDE [more details here], which provides a more visual and integrated experience

Uploading the Operating System

The operating system can be uploaded using one of the following methods:

• Open the board’s back cover, remove the SD card, upload the operating system, and then reinsert the SD card.
• Upload the operating system without opening the back cover by pressing the ‘user’ button and then inserting the USB cable into the computer.

The ‘user’ button may need to be pressed a few more times until the MaixCam appears as a USB drive labelled ‘boot’ in My Computer. Use balenaEtcher or a similar software to transfer the operating system image to the USB drive.

MaixCam Drive

Once the operating system has been transferred, remove the USB plug. The MaixCam will display the opening page on the TFT screen.

1. Tap settings, then find Wi-Fi Connections
2. Scan for the network and connect using the password
3. After connecting, the MaixCam may prompt a download for the MaixPy runtime. Follow the on-screen instructions to complete this step.
4. Explore the other icons on the TFT to familiarise yourself with the board.

MaixVision IDE and SSH Setup

• On the left side of the screen, MaixVision IDE is available
• On the right side, the MaixCam can be accessed via SSH
• From MaixVision, SSH or GFTP (Graphical FTP) can be used to interact with the board
• SSH can also be accessed directly from the command prompt using: ssh [email protected]

(Replace the IP address with the one assigned by the router.)

• When prompted for a password, enter: ‘root’
• Several sample models are available at MaixHub. However, many of these are in Chinese, and the help files are often limited. Fig. 2 shows the device setup for face recognition.

Uploading and Code Setup

Once an SSH connection has been established, files can be uploaded using scp: [scp inputfile.py [email protected]:/root/inputfile.py]. However, the most efficient method is to install MaixVision on the computer and upload files from within the IDE.

• On the right side of MaixVision IDE, open Device File Manager.
• Select the drive where the file should be uploaded and press the Upload button.

Important Note

Nothing should be uploaded to the `/boot` directory, as it may corrupt the operating system, requiring a reinstallation.

The system can learn and store new faces in real time. Each new face is assigned a unique ID and saved in the database. Simultaneously, the person.txt file is populated with the ID and a generic name. All saved faces can be cleared, and the file can be manually updated with actual names.

The MaixVision IDE interface is similar to MaixPy or OpenMV IDE, making it familiar for users.

To connect the MaixCam:

1. Click the Connect icon in the bottom-left of MaixVision IDE.
2. This establishes a connection via Wi-Fi LAN.
3. Once connected, the MaixCam can be accessed from anywhere within the local network (LAN).

Boot Start

Unlike other Sipeed boards, where code is placed in boot.py for automatic execution, MaixCam requires a different approach:

1. In MaixVision IDE, go to the bottom-left panel and select Package
2. Enter the required details: ID, name, version, developer, and description
3. Submit and install the package.

This creates a link in the `/maixapp/apps/app.info` file with the assigned name and generates a folder named `facerecognition` inside `/maixapp/apps/`, which contains three files, including main.py (the code.py file). Fig. 3 shows the Main.py file (author code snippet).

Setting Auto-start

To enable auto-start:

1. Open `/maixapp/auto_start.txt`
2. Enter: ‘facerecognition’
3. Save the file. Ensure this file contains no other entries.
4. Disconnect the MaixCam board. On reboot, the code will auto-start.

Testing

To test the system, power on the device and allow the camera to capture a person entering the frame. The system then compares the detected face against the stored database. If the face is not found, the red LED lights up, indicating ‘Unknown Person.’

However, if the face is recognised, the green LED lights up, and the gate opens for the authorised individual. Fig. 4 shows the various face repositionings.

Deploying the Device in a Small Office

The device can be deployed in a small office in the following ways: real-life deployment methods, data management, modifying or deleting entries, and a summary of key features.

Real-life deployment method

Step 1: Enroll employee faces

1. Position the camera to capture the employee’s face clearly.
2. Once the face appears sharp and focused on the screen, press Learn at the bottom-left of the TFT screen.
3. The face is saved in the system.

(This can be verified by checking the increased file size of `/root/faces.bin`.)

Step 2: Real-time operation

Once enrolled, the system functions as follows:

1. Employees stand in front of the camera.

2. If the system recognises the face from the database:

• The green LED lights up, indicating ‘Gate Open.’

• The gate mechanism activates, allowing entry.

3. If the system does not recognise the face:

• The red LED lights up, indicating ‘Gate Closed.’

Data management

The system maintains a person.txt file that maps each employee’s real name to their corresponding ID in the database.

The employee’s name and ID number are displayed on the screen during real-time operation.

Modifying or deleting entries

To modify or delete employee data the provided Python script can be used to edit entries in headless mode (SSH or serial connection). Alternatively, the person.txt file can be manually edited to update names.

Key features summary

Simple enrollment: Register faces by pressing Learn.

Automated Gate operation: Facial recognition controls access.

Visual indicators: Red/green LEDs provide clear status feedback.

Editable database: Employee records can be managed with built-in tools.

Aftermath

With advancements in AI software (for example, YOLOv5→YOLOv8), hardware capabilities are improving to handle more complex tasks.

A logical next step for this device is to develop a log register that records legitimate entries with date and timestamp. This feature can be implemented as an enhancement.

Somnath Bera (left) is an electronics enthusiast. He is a freelancer and has written several articles across the globe. Sukriti Sahoo (right) is a hardware programmer pursuing Metallurgy at Kazi Nazrul University.