Some cool application with machine learning Part 2

研究開発室の馮 志聖(マイク)です。

First I will talk about remove background use semantic segmentation on ios.

And I will extend this to AR Cut and Paste.

Second I will talk about use Real-time Text Recognition with Document Classifier on ios.

AR Cut & Paste Introduction

Sometimes we walk on street and see some cool things (like flower or animals).

And want to crop the target and didn't need the background.

Normal case : take a photo => use image processing tools remove background.

It take long time.

So I want to make it smarter.

I use semantic segmentation with deeplab model + ARKit.

And just take a photo and paste some where.

This is semantic segmentation with deeplab model.

Semantic Image Segmentation with DeepLab

https://ai.googleblog.com/2018/03/semantic-image-segmentation-with.html

AR Cut & Paste

This is structure.

AR Cut & Paste Structure

AR Cut & Paste Demo

This is simple input image and output image.

And all images is get from Pixabay.

Free-PhotosによるPixabayからの画像

Simple demo

AR Cut & Paste Final

It is cool application for create a AR photo.

This is demo for AR Cut & Paste.

AR Cut & Paste demo

Real-time Text Recognition with Document Classifier Introduction

Sometimes if we need to classify many documents or poster.

It need long time to do it.

So I try to create Real-time Text Recognition with Document Classifier.

It help us use smart way to classifier the document.

And just focus the the mobile camera on content.

It will automatic classify the content.

Real-time Text Recognition with Document Classifier

Real-time Text Recognition with Document Classifier

This is structure.

Real-time Text Recognition with Document Classifier structure

It have two main part of this application.

First is Google MLKit Text Recognition.

Google MLKit Text Recognition Text Area

https://developers.google.com/vision/android/text-overview

This is OCR system flow chart.

OCR system flow chart

https://www.researchgate.net/figure/Flow-Chart-for-OCR-system_fig4_276108387

Second is Document Classifier.

This is Document Clustering using k-means algorithm flow chart.

Document Clustering using k-means algorithm flow chart

https://www.semanticscholar.org/paper/Improved-Document-Clustering-using-k-means-Bide-Shedge/7e15f1a788e73b152ebe1ddce4a2bd92451caf62

Real-time Text Recognition with Document Classifier Demo

I use three kind of contents on Wiki for testing.

1.Wi-Fi = technology

2.Savings account = Business

3.Baseball = Sports

Real-time Text Recognition with Document Classifier Demo

Real-time Text Recognition with Document Classifier Final

Use Google MLKit Text Recognition on ios speed is very fast.

And Document Classifier is fast too.

This application can use on real-time.

It is convenient for us to find out some kind of contents.

Other

Tell us what do you think about our result , or anything else that comes to mind.

We welcome all of your comments and suggestions.

Reference

Semantic Image Segmentation with DeepLab in TensorFlow

• https://ai.googleblog.com/2018/03/semantic-image-segmentation-with.html

BackgroundRemovalWithCoreMLSample GitHub

• https://github.com/tbchen/BackgroundRemovalWithCoreMLSample

Savings account on Wiki

• https://en.wikipedia.org/wiki/Savings_account

Wi-Fi on Wiki

• https://en.wikipedia.org/wiki/Wi-Fi

Baseball on Wiki

• https://en.wikipedia.org/wiki/Baseball

Google MLKit Text Recognition on ios

• https://developers.google.com/ml-kit/vision/text-recognition/ios

DocumentClassifier

• https://github.com/toddkramer/DocumentClassifier

D. Greene and P. Cunningham. "Practical Solutions to the Problem of Diagonal Dominance in Kernel Document Clustering", Proc. ICML 2006.

• http://mlg.ucd.ie/files/publications/greene06icml.pdf