フレクトのクラウドblog re:newal

http://blog.flect.co.jp/cloud/からさらに引っ越しています

Some cool application with machine learning Part 2

研究開発室の馮 志聖(マイク)です。

First I will talk about remove background use semantic segmentation on ios.

And I will extend this to AR Cut and Paste.

Second I will talk about use Real-time Text Recognition with Document Classifier on ios.

AR Cut & Paste Introduction

Sometimes we walk on street and see some cool things (like flower or animals).

And want to crop the target and didn't need the background.

Normal case : take a photo => use image processing tools remove background.

It take long time.

So I want to make it smarter.

I use semantic segmentation with deeplab model + ARKit.

And just take a photo and paste some where.

This is semantic segmentation with deeplab model.

f:id:fengchihsheng:20201228115149p:plain
Semantic Image Segmentation with DeepLab

https://ai.googleblog.com/2018/03/semantic-image-segmentation-with.html

AR Cut & Paste

This is structure.

f:id:fengchihsheng:20201228115311p:plain
AR Cut & Paste Structure

AR Cut & Paste Demo

This is simple input image and output image.

And all images is get from Pixabay.

Free-PhotosによるPixabayからの画像

f:id:fengchihsheng:20210104131402p:plain
Simple demo

AR Cut & Paste Final

It is cool application for create a AR photo.

This is demo for AR Cut & Paste.

f:id:fengchihsheng:20210104125653g:plain
AR Cut & Paste demo

Real-time Text Recognition with Document Classifier Introduction

Sometimes if we need to classify many documents or poster.

It need long time to do it.

So I try to create Real-time Text Recognition with Document Classifier.

It help us use smart way to classifier the document.

And just focus the the mobile camera on content.

It will automatic classify the content.

f:id:fengchihsheng:20210104132505p:plain
Real-time Text Recognition with Document Classifier

Real-time Text Recognition with Document Classifier

This is structure.

f:id:fengchihsheng:20201229005213p:plain
Real-time Text Recognition with Document Classifier structure

It have two main part of this application.

First is Google MLKit Text Recognition.

f:id:fengchihsheng:20201229111312p:plain
Google MLKit Text Recognition Text Area

https://developers.google.com/vision/android/text-overview

This is OCR system flow chart.

f:id:fengchihsheng:20201229111730p:plain
OCR system flow chart

https://www.researchgate.net/figure/Flow-Chart-for-OCR-system_fig4_276108387

Second is Document Classifier.

This is Document Clustering using k-means algorithm flow chart.

f:id:fengchihsheng:20201229112755p:plain
Document Clustering using k-means algorithm flow chart

https://www.semanticscholar.org/paper/Improved-Document-Clustering-using-k-means-Bide-Shedge/7e15f1a788e73b152ebe1ddce4a2bd92451caf62

Real-time Text Recognition with Document Classifier Demo

I use three kind of contents on Wiki for testing.

1.Wi-Fi = technology

2.Savings account = Business

3.Baseball = Sports

f:id:fengchihsheng:20201229002734g:plain
Real-time Text Recognition with Document Classifier Demo

Real-time Text Recognition with Document Classifier Final

Use Google MLKit Text Recognition on ios speed is very fast.

And Document Classifier is fast too.

This application can use on real-time.

It is convenient for us to find out some kind of contents.

Other

Tell us what do you think about our result , or anything else that comes to mind.

We welcome all of your comments and suggestions.

Reference

Semantic Image Segmentation with DeepLab in TensorFlow
BackgroundRemovalWithCoreMLSample GitHub
Savings account on Wiki
Wi-Fi on Wiki
Baseball on Wiki
Google MLKit Text Recognition on ios
DocumentClassifier
D. Greene and P. Cunningham. "Practical Solutions to the Problem of Diagonal Dominance in Kernel Document Clustering", Proc. ICML 2006.