Using deep learning parfor to predict in parfor get GPU out of memory issues

I have deep learning model trained and now want to use it for predict on a batch of my images.

I have a 12GB GPU and 16 physical cores cpu and want to use parfor for predict function. The psedo code looks like below

parfor i = 1:100

image = read(‘xxx’);

mask = predict(net, image);

end

However, I got gpu out of memory issues when use parfor. Just use for works fine. Also, forecefully turn off all gpu by disable them in device manager works as well to run on 12 core CPU with parfor, which is even faster to use for with GPU (I think because copying single image between devices is even more expensive).

Anyone has idea how to use both parfor and gpu to for deep learning inference? I also try to limit the number of the worker in parfor to 2 and even 1. Even 1 worker gives the same gpu out of memeory problem. I thus assume the parfor doesn’t realse the memory or resource timely after each prediction. I even added the gpuDevice command after predict trying to forcefully clear out memory, but the error stays.

ANSWER

Matlabsolutions.com provide latest MatLab Homework Help,MatLab Assignment Help for students, engineers and researchers in Multiple Branches like ECE, EEE, CSE, Mechanical, Civil with 100% output.Matlab Code for B.E, B.Tech,M.E,M.Tech, Ph.D. Scholars with 100% privacy guaranteed. Get MATLAB projects with source code for your learning and research.

1. Set the ‘ExecutionEnvironment’to ‘cpu’ in your predict function. Documentation on this can be found here.

2. Using parfor with a single GPU will not work. Using parfor with 16 workers will try to access the 1 GPU in parallel 16 times.

3. It is strange that you are receiving an out of memory error but this maybe due to the fact you are using predict with parfor.

I would recommend performing a small test on 10% of your images to find which method runs quicker:

1. Predict all images using the single GPU. Put a tic/toc around it to test the timing.

2. Predict all images using parfor with the ExecutionEnvironment set to cpu. Use tic/toc again to test the timing.

SEE COMPLETE ANSWER CLICK THE LINK

--

--

--

Simple! That is me, a simple person. I am passionate about knowledge and reading. That’s why I have decided to write and share a bit of my life and thoughts to.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Transfer Learning to solve a Classification Problem

How to run Keras model on Movidius neural compute stick

Tweet Sentiment Extraction

Introduction to Artificial Neural Network

Emotion Recognition From Speech

COMPREDICT Platform Architecture — Part 1: AI Core & ML Framework

Zero Shot Super Resolution Part 3: Training the Model -

Timing memory recall

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Technical Source

Technical Source

Simple! That is me, a simple person. I am passionate about knowledge and reading. That’s why I have decided to write and share a bit of my life and thoughts to.

More from Medium

How to decide dimension (using correlationDimension function)?

Essential Libraries To Have In Your Tool For Data Science And ML — Series #1 — NumPy

This model will make you understand how you can apply Machine Learning to the retail industry

Detecting Room Occupancy Using Machine Learning and Sensor Data