First I will talk about some cool application on Tensorflow Lite.
Style Transfer Introduction
Name is style transfer.
And it can work on mobile.
Apply any styles on an input image to create a new artistic image.
It can use on take artistic photo and post on social media like twitter.
Share the cool photo to other person.
This is structure.
Style Transfer Demo
It use pre-train model for prediction.
Style Transfer Final
It is cool application for create a new artistic image.
And speed is very fast.
I will use same image and same style for testing.
This is some results.
It have 4 steps.
1.Preprocessing. 2.Style prediction. 3.Style transform. 4.Post-processing.
At last these are all processing run on CPU and GPU.
This is Demo for iphone xs.
Run on iphone xs need 490-500 milliseconds for each frame.
FPS nearby 2.
It is slow.
I try to create more fast demo for Style transform.
Preprocessing is heavy.
So I try to made it more fast.
I use camera buffer only to next step.
Not need any preprocessing.
These are all processing after fix run on CPU and GPU.
On these benchmark we can see iphone 11 CPU is the fastest.
More fast demo like this image.
Run on iphone xs need 150 milliseconds for each frame.
FPS nearby 6.6
YOLACT: Real-time Instance Segmentation.
I think maybe someone know about Semantic Segmentation.
Semantic Segmentation is image classification and localization.
Instance Segmentation is object detection and Semantic Segmentation.
Like these images.
This is structure of YOLACT.
This is table of speed on pc.
In this graph YOLACT is in Real-time area.
This graph don't have mobile version.
Because some of them not support work on mobile.
So how YOLACT speed work on ios?
How to use YOLACT model on CoreML?
Follow this URL.
And run onnx_to_coreml.py on this github.
After finished it put on xcode project and use it.
This is demo run on iphone xs.
It only have 1.5-1.8 FPS.
Use YOLACT on ios speed is very fast.
But it can not real-time on mobile.
Real-time need FPS more than 30.
I think it can only use on some business case not need real-time.
Business case like scan the multi barcode.
Tell us what do you think about our result , or anything else that comes to mind.
We welcome all of your comments and suggestions.