Deep Face Recognition with Redis

Key value databases come with a high speed and performance where we mostly cannot reach in relational databases. Herein similar to Cassandra, Redis is a fast key value store solution. In this post, we are going to adopt Redis to build an overperforming face recognition application. On the other hand, this could be adapted to NLP studies or any reverse image search case such as in Google Images.

Vlog

You can either watch the following video or keep reading this tutorial. They both cover the deep face recognition with redis and python.

---

🙋‍♂️ You may consider to enroll my top-rated machine learning course on Udemy

---

Vector Similarity Search with k-NN in Redis

Installations

The official redis distribution is available for Linux and MacOS here. Besides, it offers a readt to use Redis Cloud alternative. However, it has no official release for Windows OS. Luckily, Microsoft Team published Windows executables here.

I tested the mac distribution of redis. You can easily install it with homebrew. Then, running redis-server command gets redis server up and calling redis-cli open its command line interface and you can communicate with redis server here.

brew install redis

I also tested Redis server for Windows with Redis-x64-3.0.504.zip package in this post. I just unzip the archieve file, then redis-server.exe to get Redis server up and redis-cli.exe to use its command line interface if necessary. Running redis-server gets Redis server up in 6379 port. I will communicate with Redis server on this port later.

Once your Redis server is up, then you can run some tests with its command line interface. Redis is basically a key value database. We can store key value pairs with set command whereas we can read the key values with get command.

Redis client

We will communicate to Redis with Python. This redis package on pip provides a python interface. Then, we will store key value pairs and read the values of some specific keys within Python.

#!pip install redis
import redis
redis = redis.StrictRedis(host='localhost', port=6379, db=0)

Face recognition pipeline

A modern face recognition pipeline consists of 4 common stages: detect, align, represent and verify. I highly recommend you to watch the following video to understand how facial recognition works. Luckily, all of those common stages are covered in deepface framework for python.

Herein, we will represent our facial database as vector with deepface library. This requires to apply pre-processing stages: detect and align as well. Then we will store those representations in Redis key value store. Verification task could be handled on Redis (server side). We can alternatively retrieve the representations from Redis to Python first and then handle verification in client side.

---

---

Deepface wraps several state-of-the-art face recongition models: VGG-Face, Google FaceNet, OpenFace, Facebook DeepFace, DeepID, Dlib and ArcFace. In this study, we are going to use FaceNet model to represent facial images as vectors.

FaceNet, VGG-Face and ArcFace overperform among others. Here, you can watch how to determine the best model.

Representations

We will use the unit test items of deepface library as a facial database.

#Ref: https://github.com/serengil/deepface/tree/master/tests/dataset
local_db = {
'angelina': 'deepface/tests/dataset/img2.jpg',
'jennifer': 'deepface/tests/dataset/img56.jpg',
'scarlett': 'deepface/tests/dataset/img49.jpg',
'katy': 'deepface/tests/dataset/img42.jpg',
'marissa': 'deepface/tests/dataset/img23.jpg'
}

We will read those images first, detect faces and align secondly, represent them as vectors thirdly, and finally store them in Redis.

from deepface.commons import functions
from tqdm import tqdm

identities = list(local_db.keys())

for i in tqdm(range(0, len(identities))):
    name = identities[i]
    img_path = local_db[name]
    
    embedding = DeepFace.represent(img_path = img_path, model_name = "Facenet")[0]["embedding"]
    
    #store in redis
    redis.rpush("embedding:"+name, *embedding)
    redis.set("photo:"+name, img_path)

We will store the exact image path with photo:angelina key and vector representation with embedding:angelina for Angelina Jolie.

DeepFace wraps several face detectors: opencv, ssd, mtcnn and dlib. MTCNN is the most robust one but SSD is the fastest.

Here, you can watch how those detectors performing on real time video.

Herein, retinaface is the cutting-edge technology for face detection. It can even detect faces in the crowd. Besides, it finds some facial landmarks including eye coordinates. In this way, its alignment score is high as well.

Target image

We’ve already stored several representations in Redis. Let’s represent a new one and look for its identity in our Redis database. Notice that this target image does not exist in Redis as is.

target_img_path = "target.png"

#detect and align
target_img = DeepFace.extract_faces(img_path = target_img_path)[0]["face"]

#represent
target_embedding = DeepFace.represent(img_path = target_img_path, model_name = "Facenet")[0]["embedding"]

Client side verification

We have the representation of the target image in the client side already. Representations of facial database are stored in Redis as well. Let’s retrieve the representation of an identity from Redis to client side. Notice that Redis is a key value store and this approach overperforms than server side approach.

---

---

def verify_face(key):
    embedding = redis.lrange('embedding:'+key, 0, -1)
    
    distance = findEuclideanDistance(target_embedding, np.array(embedding).astype('float'))
    
    if distance <= 10:
        print("this is "+key)
    else:
        print("this IS NOT "+key)

verify_face('angelina')

Proposed system actually doesn’t know the identity of target image (even though we know it is Angelina Jolie). lrange(’embedding:angelina’,0 , -1) command will return the embedding of Angelina Jolie. We then find the distance between target one and Angelina in the client side. If the distance value is less than or equal to 10, then that pair is same person. Here, the threshold value for FaceNet model and Euclidean distance pair is tuned in deepface.

Server side verification

As I mentioned before, Redis is a key value database and it should be responsible for returning the result of a key. I mean that we should pass the vector embedding of target one and an identity name (e.g. angelina) to to Redis. Redis then find the embedding of that identity and compare it to the target one in the server side.

We have to write lua scripts to handle some logics in the server side. Here, 1 index argument is the name of an identity (e.g. angelina), 2 to 129 index values are the target embedding (because FaceNet represents facial images as 128 dimensional vectors).

verify_script = '''
local source = KEYS[1]
local source_embedding = redis.call("lrange", "embedding:"..source, 0, -1)

local distance = 0
for i = 1, #source_embedding do
distance = distance + (source_embedding[i] - KEYS[1+i])^2
end

distance = math.sqrt(distance)

return {"euclidean_distance", tostring(distance)}
'''

command = redis.register_script(verify_script)

float(command(keys = ["angelina", *target_embedding])[1].decode()) <= 10

float(command(keys = ["jennifer", *target_embedding])[1].decode()) <= 10

This will return the distance value for an identity. We then decidide they are same person or different persons based on the threshold.

We can alternatively save this script in a file with .lua extension and call it with the following command.

redis-cli --eval verify-face.lua angelina 1 2 3 … 128

No matter the size of your facial database. We will access the embedding of a specific key each time. That’s why, this overperforms!

Server side face recognition

That would not be a best practice but what if we look for the target one in our facial database? This requires to apply face verification several times. We will retrieve all keys starting with embedding* expression first. Then, iterate over those embeddings and find the distance with the target one. If the distance is less than the threshold, then we will append it to the return object.

find_script = '''
local db = redis.call("SCAN", 0, "MATCH", "embedding:*")[2]

local identities = {}
local distances = {}

local idx = 1
for i=1, #db do
    local source_key = db[i]
    local source_embedding = redis.call("lrange", source_key, 0, -1)
    
    local distance = 0
    for j=1, #source_embedding do
        distance = distance + (source_embedding[j] - KEYS[j])^2
    end
    
    distance = math.sqrt(distance)
    
    if distance <= 10 then
        distances[idx] = tostring(distance)
        identities[idx] = source_key
        idx = idx + 1
    end
    
end

return {identities, distances}
'''

command = redis.register_script(find_script)
results = command(keys = [*target_embedding])
print(results)

This returns the found identity in the facial database.

[[b’embedding:angelina’], [b’7.091028809737′]]

As I mentioned before, Redis is a key value database and this server side face recognition approach is not optimal for really large scale data sets.

---

---

Lightweight way

If you need a more basic design, then deepface itself offers verify function for face verification task.

Deepface offers a find function as well to handle face recognition task.

Large scale face recognition

If your facial database has millions level data, and your task requires to search an identity in all database, you should switch your approach. Hadoop, Mongo or Cassandra or Redis are no sql solutions and they are as strong as the hardware you have. Those applies k-nn algorithm basically. On the other hand, approximate nearest neighbor algorithm reduces your key space dramatically. Spotify Annoy, Facebook Faiss, NMSLIB or Elasticsearch covers a-nn algorithm.

Tech Stack Recommendations

Face recognition is mainly based on representing facial images as vectors. Herein, storing the vector representations is a key factor for building robust facial recognition systems. I summarize the tech stack recommendations in the following video.

Conclusion

So, we have adopted Redis key value store to build a face recognition pipeline in this post. Key value databases come with a high speed and we mostly cannot have same level performance with relational databases. If your task is mainly based on face verification, key value stores would be the best option for your case.

I pushed the source code of this study to GitHub. You can support this study if you star🙏 the repo.

---

Like this blog? Support me on Patreon

---