Flask is a pretty web framework to publish python projects as an api. Recently, I’ve published one of my project as an API with flask. There are lots of tutorials how to publish rest services with flask. Today, I would like to share my experiments with flask for a machine learning project.
The logo of the flask framework looks like a horn flask. It is similar to horn of Angelina Jolie in Maleficent.
🙋♂️ You may consider to enroll my top-rated machine learning course on Udemy
Vlog
This blog post covers the experiments I had during building the API explained in the following video.
Passing already built models
Building complex machine learning models and loading pre-trained weights are costly. That’s why, I plan to build my deep learning models when api gets up. In this way, these complex models will be built once and I can make predictions with thouse already built models.
Herein, verify is an operation of my service. It will call verify function of DeepFace interface. Verify function of DeepFace interface will make predictions in the background. I need to pass a pre-built model to verify operation of the service and then pass the model function I consume.
app = Flask(__name__)
my_model = VGGFace.loadModel()
@app.route('/verify', methods=['POST'])
def verify():
resp_obj = DeepFace.verify(instances, model = my_model)
When I designed this structure, I had the following error.
ValueError: Tensor Tensor(“flatten_1/Reshape:0”, shape=(?, ?), dtype=float32) is not an element of this graph.
This is because flask runs multiple threads. Tensorflow model is loaded in one thread and we try to use that tensorflow model in another thread. I spent a long time to solve this issue but the solution is easy. I stored the default graph in the same level of model building. Besides, in the service operation I defined this graph object as global (this is not a must). Finally, I called the function responsible for making predictions in the with graph.as_default() block.
app = Flask(__name__)
my_model = VGGFace.loadModel()
graph = tf.get_default_graph()
@app.route('/verify', methods=['POST'])
def verify():
global graph #not must
with graph.as_default():
resp_obj = DeepFace.verify(instances, model = my_model)
return resp_obj
In my case, model building lasts 25 seconds but making predictions lasts 600ms if you have the pre-built model. I need to re-build the model for each request if I wouldn’t pass pre-trained model to DeepFace.verify function.
Calling just tensorflow / keras related functions
The use case mentioned in the previous section might be unique. You might not need to pass pre-trained models. The function you are calling builds a tensorflow / keras model and make prediction in the background.
Herein, I will call DeepFace.verify and this function will create a Keras Sequential model and make predictions.
app = Flask(__name__)
@app.route('/verify', methods=['POST'])
def verify():
resp_obj = DeepFace.verify(instances)
return resp_obj
This time, I had the following error.
TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor(“Placeholder_1:0”, shape=(64,), dtype=float32) is not an element of this graph.
To solve this problem, you should modify the content of DeepFace.verify. I need to call clear session command of Keras Backend module before building model and at the end of the function as well.
#This is DeepFace.py from keras import backend as K def verify(instances): K.clear_session() model = VGGFace.loadModel() img_representation = model.predict(instances[0])[0,:] K.clear_session() return resp_obj
Combination of these two cases is dangerous
I try to combine these two cases. I mean that I firstly call DeepFace.verify function in the with graph.as_default(): block and I call clear_session in DeepFace.verify as well. The exception is interesting.
AssertionError: Do not use tf.reset_default_graph() to clear nested graphs. If you need a cleared graph, exit the nesting and create a new graph.
So, you must not clear session if you are working in default graph.
Conclusion
So, we’ve mentioned flask for python in the perspective of a machine learning project.
Here, you can find the project implementation.
Like this blog? Support me on Patreon
