00:00:00.001 --> 00:00:02.540

Does a computer see in color or black and white?

00:00:02.540 --> 00:00:06.700

It's time to find out on episode 11 of Talk Python To Me

00:00:06.700 --> 00:00:08.960

with our guest, Adrian Rosebrock,

00:00:08.960 --> 00:00:12.040

recorded Thursday, May 20th, 2015.

00:00:12.040 --> 00:00:42.020

Welcome to Talk Python To Me.

00:00:42.020 --> 00:00:45.160

A weekly podcast on Python, the language, the libraries,

00:00:45.160 --> 00:00:47.000

the ecosystem, and the personalities.

00:00:47.000 --> 00:00:48.840

This is your host, Michael Kennedy.

00:00:48.840 --> 00:00:51.380

Follow me on Twitter where I'm @mkennedy

00:00:51.380 --> 00:00:54.000

and keep up with the show and listen to past episodes

00:00:54.000 --> 00:00:55.900

at talkpythontome.com.

00:00:55.900 --> 00:00:58.820

This episode, we'll be talking with Adrian Rosebrock

00:00:58.820 --> 00:01:03.240

about computer vision, OpenCV, and PyImage Search.

00:01:03.240 --> 00:01:05.740

Hello, everyone.

00:01:05.740 --> 00:01:08.780

I have a bunch of cool news and announcements for you this week.

00:01:08.940 --> 00:01:11.960

First, this show on PyImage Search

00:01:11.960 --> 00:01:13.760

is a listener-suggested show.

00:01:13.760 --> 00:01:16.180

Thank you to J.I. Lorenzetti

00:01:16.180 --> 00:01:18.660

for reaching out to me and suggesting this topic.

00:01:18.660 --> 00:01:21.100

You can find his contact details in the show notes.

00:01:21.100 --> 00:01:24.900

As always, I'm also excited to be able to tell you

00:01:24.900 --> 00:01:27.680

that this episode is brought to you by CodeShip.

00:01:27.680 --> 00:01:30.980

CodeShip is a platform for continuous integration

00:01:30.980 --> 00:01:33.060

and continuous delivery as a service.

00:01:33.060 --> 00:01:36.000

Please take a moment and check them out at codeship.com

00:01:36.000 --> 00:01:38.440

or follow them on Twitter where they're at codeship.

00:01:38.440 --> 00:01:42.020

Did you know that most of our shows come with full transcripts

00:01:42.020 --> 00:01:43.900

and a cool little search filter feature?

00:01:43.900 --> 00:01:46.480

If you're looking for something you hear in an episode,

00:01:46.480 --> 00:01:49.000

just click the full transcript button on the episode page

00:01:49.000 --> 00:01:49.960

and search for it.

00:01:49.960 --> 00:01:52.360

Also, I want to say thank you to everyone

00:01:52.360 --> 00:01:55.180

who has been participating in the conversation on Twitter

00:01:55.180 --> 00:01:56.840

where we're at Talk Python.

00:01:57.580 --> 00:02:00.140

It's a great feeling to see all the feedback and thoughts

00:02:00.140 --> 00:02:01.700

every week when we release a new show.

00:02:01.700 --> 00:02:04.420

But if you have something more nuanced to say

00:02:04.420 --> 00:02:06.380

that doesn't fit in 140 characters

00:02:06.380 --> 00:02:09.420

or you want it to be more permanent than Twitter,

00:02:09.420 --> 00:02:13.560

every episode page has a Discus comment section at the bottom.

00:02:13.560 --> 00:02:15.140

I encourage you to post your thoughts there.

00:02:15.140 --> 00:02:18.880

This week, I ran across a really awesome GitHub project

00:02:18.880 --> 00:02:20.520

called Python-Patterns.

00:02:20.520 --> 00:02:22.700

You can find it at GitHub.com

00:02:22.700 --> 00:02:26.460

slash f-a-i-f slash python dash patterns.

00:02:26.460 --> 00:02:29.900

It's a collection of really crisp design patterns

00:02:29.900 --> 00:02:31.760

implemented in a Pythonic manner.

00:02:31.760 --> 00:02:34.060

For example, you'll find patterns such as

00:02:34.060 --> 00:02:37.100

the adapter, builder, chain, decorator, facade,

00:02:37.100 --> 00:02:39.260

and flyweight patterns, just to name a few.

00:02:39.260 --> 00:02:41.180

It's really extensive and pretty cool.

00:02:41.180 --> 00:02:42.860

I think you'll learn something if you check it out.

00:02:42.860 --> 00:02:45.900

Finally, I put together a cool YouTube playlist.

00:02:45.900 --> 00:02:50.620

This is a series of nine lectures from Dr. Philip Gao,

00:02:50.620 --> 00:02:53.240

a professor at the University of Rochester, New York.

00:02:53.240 --> 00:02:58.520

Find him on Twitter where he's at P-G-B-O-V-I-N-E, P-G-Bovine.

00:02:58.520 --> 00:03:01.960

The video series is entitled C Internals,

00:03:01.960 --> 00:03:05.800

a 10-hour code walk through the Python interpreter source code.

00:03:05.800 --> 00:03:10.520

You can find it at bit.ly.com slash cpythonwalk,

00:03:10.520 --> 00:03:12.720

all lowercase, no spaces.

00:03:12.720 --> 00:03:15.580

Also, you'll find all these links in the show notes.

00:03:16.520 --> 00:03:18.780

Now, let's get to the interview with Adrian.

00:03:18.780 --> 00:03:23.800

Let me introduce Adrian.

00:03:23.800 --> 00:03:28.460

Adrian Rosebrock is an author and blogger at pyimagesearch.com.

00:03:28.460 --> 00:03:32.800

He has a PhD in computer science with a focus on computer vision and machine learning

00:03:32.800 --> 00:03:36.020

and has been studying computer vision his entire adult life.

00:03:36.020 --> 00:03:41.720

He has consulted for the National Cancer Institute to develop methods to predict breast cancer risks

00:03:41.720 --> 00:03:48.080

using breast histology images and authored a book, Practical Python and OpenCV,

00:03:48.080 --> 00:03:52.400

on utilizing Python and OpenCV to build real-world computer vision applications.

00:03:52.400 --> 00:03:56.200

Adrian, welcome to the show.

00:03:56.200 --> 00:03:57.060

Oh, thank you.

00:03:57.060 --> 00:03:58.000

It's great to be here.

00:03:58.180 --> 00:04:04.420

I'm very excited about computer vision and sort of merging the real world with computer science,

00:04:04.420 --> 00:04:05.320

with robotics.

00:04:05.320 --> 00:04:07.920

And I think there's just some really neat stuff going on.

00:04:07.920 --> 00:04:10.320

And you're doing a very cool part in that.

00:04:10.320 --> 00:04:11.120

Oh, thank you.

00:04:11.120 --> 00:04:13.340

So we're going to talk about PyImageSearch.

00:04:13.340 --> 00:04:18.600

We're going to talk about OpenCV and some of the challenges and even the future of these types of technologies.

00:04:18.600 --> 00:04:22.940

But before we get there, you know, everyone's interested in how people got started in programming and Python.

00:04:22.940 --> 00:04:23.680

What's your story?

00:04:23.680 --> 00:04:27.400

I started programming when I was in high school.

00:04:27.400 --> 00:04:33.380

I started out with the basics of HTML, JavaScript, CSS, did some basic programming.

00:04:33.380 --> 00:04:36.140

And, you know, I'm probably getting a lot of hate mail about this.

00:04:36.140 --> 00:04:41.960

But when I first started learning how to program, I did not like the Python programming language that much.

00:04:41.960 --> 00:04:45.560

And this was around the early version 2 of Python.

00:04:45.740 --> 00:04:47.760

I didn't like the syntax.

00:04:47.760 --> 00:04:49.420

I didn't like the white space.

00:04:49.420 --> 00:04:52.880

And for a long time, I was really, really put off by Python.

00:04:52.880 --> 00:04:55.640

And that was a huge mistake on my part.

00:04:55.640 --> 00:04:57.120

I don't know what was wrong with me back then.

00:04:57.120 --> 00:04:59.340

I guess it was just high school ignorance or something.

00:04:59.340 --> 00:05:04.620

But by the time I got to college, I started working in Python a lot more.

00:05:04.620 --> 00:05:07.500

And that's especially true in the scientific area.

00:05:07.500 --> 00:05:15.340

You see all these incredible packages in Python like NumPy and SciPy that just integrate with computer vision and machine learning.

00:05:15.660 --> 00:05:17.480

And all other types of libraries.

00:05:17.480 --> 00:05:25.060

And more and more people were transitioning over from languages like MATLAB to languages like Python.

00:05:25.060 --> 00:05:26.440

And that's so cool.

00:05:26.440 --> 00:05:30.580

And it really wasn't until college that I got into Python.

00:05:30.580 --> 00:05:32.740

And I remember this one girl.

00:05:32.740 --> 00:05:34.260

She was in my machine learning class.

00:05:34.260 --> 00:05:38.740

And she had a sticker on the back of her laptop that said, Python will save the world.

00:05:38.900 --> 00:05:40.780

I don't know how, but it will.

00:05:40.780 --> 00:05:42.460

And that resonated with me.

00:05:42.460 --> 00:05:44.600

I'm like, that sticker's true.

00:05:44.600 --> 00:05:46.540

That is absolutely true.

00:05:46.540 --> 00:05:48.520

It's such a great language.

00:05:48.520 --> 00:05:52.060

So unfortunately, I did not have the best first experience with Python.

00:05:52.060 --> 00:05:55.900

It took me four or five years later to actually come around.

00:05:56.120 --> 00:05:58.560

But now that I'm here, I love it.

00:05:58.560 --> 00:06:01.200

And I can't imagine programming in any other language.

00:06:01.200 --> 00:06:07.320

It's almost a freeing feeling, a relaxing zen when you're coding in Python.

00:06:07.320 --> 00:06:09.560

Yeah, that's a funny story.

00:06:09.560 --> 00:06:11.360

It really is a wonderful language.

00:06:11.940 --> 00:06:13.840

I also took a while to get there.

00:06:13.840 --> 00:06:16.960

But looking back, I would have enjoyed being there sooner.

00:06:16.960 --> 00:06:21.020

I went from MATLAB to C++ on Silicon Graphics machines.

00:06:21.020 --> 00:06:23.960

So I had a bit of a torturous introduction.

00:06:23.960 --> 00:06:24.760

But it was all good.

00:06:24.760 --> 00:06:41.120

CodeShip is a hosted, continuous delivery service focused on speed, security, and customizability.

00:06:41.260 --> 00:06:46.920

You can set up continuous integration in a matter of seconds and automatically deploy when your tests have passed.

00:06:46.920 --> 00:06:50.160

CodeShip supports your GitHub and BitBucket projects.

00:06:50.160 --> 00:06:52.940

You can get started with CodeShip's free plan today.

00:06:52.940 --> 00:06:55.080

Should you decide to go with a premium plan,

00:06:55.080 --> 00:07:01.460

Talk Python listeners can save 20% off any plan for the next three months by using the code TALKPYTHON.

00:07:01.460 --> 00:07:03.160

All caps, no spaces.

00:07:03.160 --> 00:07:05.800

Check them out at CodeShip.com.

00:07:05.800 --> 00:07:10.500

And tell them thanks for sponsoring the show on Twitter where they're at CodeShip.

00:07:10.580 --> 00:07:24.200

So you're focused on computer vision and image processing.

00:07:24.200 --> 00:07:25.900

Where did that story begin?

00:07:25.900 --> 00:07:29.120

That story also started in high school.

00:07:29.360 --> 00:07:33.680

Originally, I had this idea that I wanted to go work for Adobe.

00:07:33.680 --> 00:07:37.740

And I wanted to work on developing Photoshop and Illustrator.

00:07:37.740 --> 00:07:42.660

I love the idea of being able to write code that could analyze an image.

00:07:42.660 --> 00:07:46.820

And for whatever reason, that just like really captured my imagination.

00:07:47.240 --> 00:07:49.920

I could see these algorithms running in Photoshop.

00:07:49.920 --> 00:07:52.860

And I was like, you know, what's really going on behind the scenes?

00:07:52.860 --> 00:07:54.520

Like, how are they manipulating these images?

00:07:54.520 --> 00:07:56.020

What does this code look like?

00:07:56.020 --> 00:08:01.880

So for the longest time, I really wanted to develop these graphic editing applications.

00:08:02.280 --> 00:08:07.080

But I didn't have the math experience.

00:08:07.080 --> 00:08:11.600

This may surprise some people, given that I have a PhD in computer science.

00:08:11.600 --> 00:08:16.760

But up until late high school, I did not do well in mathematics courses.

00:08:17.320 --> 00:08:22.400

I got C's in algebra and geometry.

00:08:22.400 --> 00:08:27.180

And it really wasn't until I kind of really put my back against the wall.

00:08:27.180 --> 00:08:28.480

And I said, you know what?

00:08:28.480 --> 00:08:30.780

I got to learn calculus and statistics.

00:08:30.780 --> 00:08:33.420

So I did a self-study in AP Calc.

00:08:33.420 --> 00:08:34.600

And I took AP statistics.

00:08:34.600 --> 00:08:36.280

And I did well with those.

00:08:36.360 --> 00:08:38.080

I'm like, man, math is fun now.

00:08:38.080 --> 00:08:40.040

Like, I understand this.

00:08:40.040 --> 00:08:41.980

So I got to college.

00:08:41.980 --> 00:08:47.920

And I only took one computer vision course at the, because the school I went to didn't really

00:08:47.920 --> 00:08:49.380

have a computer vision focus.

00:08:49.380 --> 00:08:53.220

They had a wonderful machine learning focus, but not really a computer vision focus.

00:08:53.220 --> 00:09:00.200

And what I found out was that, you know, you don't need a mathematical background to get

00:09:00.200 --> 00:09:01.280

started in computer vision.

00:09:01.280 --> 00:09:05.480

And I think this is true in a lot of areas of computer science, whether or not people want

00:09:05.480 --> 00:09:06.140

to admit it.

00:09:06.140 --> 00:09:11.260

A lot of people talk themselves out of getting started and stuff, especially challenging things,

00:09:11.260 --> 00:09:13.320

because they're just scared of it.

00:09:13.320 --> 00:09:14.180

They don't want to fail.

00:09:14.180 --> 00:09:17.700

And that's the cool thing about Python.

00:09:17.700 --> 00:09:20.140

Like, you almost don't have to worry about the code.

00:09:20.140 --> 00:09:23.260

You get to focus on learning a new skill.

00:09:23.260 --> 00:09:25.300

And for me, that was computer vision.

00:09:25.300 --> 00:09:30.680

And that was the OpenCV library, an open source library that makes working with computer vision

00:09:30.680 --> 00:09:31.520

a lot easier.

00:09:31.520 --> 00:09:36.120

So again, it really wasn't until college that I really started to get into it.

00:09:36.120 --> 00:09:37.020

getting interested.

00:09:37.020 --> 00:09:38.680

Or not necessarily getting interested.

00:09:38.680 --> 00:09:42.320

More so being able to take action on what I wanted to do.

00:09:42.320 --> 00:09:43.060

Right.

00:09:43.060 --> 00:09:45.660

Maybe, you know, it felt a little unattainable.

00:09:45.660 --> 00:09:48.480

Like, I'm going to go be an engineer, but I don't know math.

00:09:48.480 --> 00:09:49.860

And so there's no way I can do this.

00:09:49.860 --> 00:09:53.120

But once you kind of got over that hump, then it was no big deal, right?

00:09:53.120 --> 00:09:53.820

Right.

00:09:53.820 --> 00:09:54.720

Yeah.

00:09:54.720 --> 00:09:55.760

That's very freeing.

00:09:55.860 --> 00:10:00.000

So you mentioned OpenCV and your project is PyImageSearch.

00:10:00.000 --> 00:10:01.260

What's the relationship there?

00:10:01.260 --> 00:10:08.640

So OpenCV, again, it's a computer vision library that makes working with images a lot easier.

00:10:08.640 --> 00:10:14.920

You know, it abstracts the code that loads an image off of disk or does edge detection or

00:10:14.920 --> 00:10:19.720

thresholding or any other simple image processing function like that.

00:10:19.780 --> 00:10:23.580

allows you to actually build complicated computer vision programs.

00:10:23.580 --> 00:10:28.580

You can do things like tracking objects and images or video streams, for example.

00:10:28.580 --> 00:10:29.460

Detecting faces.

00:10:29.460 --> 00:10:31.480

Recognizing whose face it is.

00:10:31.480 --> 00:10:34.960

And OpenCV really facilitates this process.

00:10:35.940 --> 00:10:41.900

And OpenCV really is like the de facto library for computer vision and image processing.

00:10:41.900 --> 00:10:44.820

And you have bindings for it in countless languages.

00:10:44.820 --> 00:10:52.480

The library itself is written in C and C++, but you can get bindings and access it in Java

00:10:52.480 --> 00:10:55.620

and any of the .NET frameworks in Python.

00:10:57.240 --> 00:11:03.820

So again, while you can access it in a programming language, I have this love for Python now.

00:11:03.820 --> 00:11:10.000

And when I took the course, the computer vision course in college, I realized, man, like, people

00:11:10.000 --> 00:11:14.180

are spending a lot of time writing their class projects in C and C++.

00:11:14.180 --> 00:11:15.320

Why are they doing that?

00:11:15.320 --> 00:11:18.980

Like, you're fighting over these weird compile time errors.

00:11:18.980 --> 00:11:21.180

And, you know, you're not really learning anything.

00:11:21.680 --> 00:11:24.680

And that's kind of the tenet behind PyImageSearch.

00:11:24.680 --> 00:11:30.220

It's a blog that I run dedicated to teaching computer vision, image processing, and OpenCV using

00:11:30.220 --> 00:11:31.600

the Python programming language.

00:11:31.600 --> 00:11:32.140

That's great.

00:11:32.140 --> 00:11:36.040

And I think that, you know, using Python seems like the perfect choice.

00:11:36.040 --> 00:11:41.920

You're sort of orchestrating these high-level functions that are calling down into C++, doing

00:11:41.920 --> 00:11:43.740

high-performance stuff, and then giving you the answer.

00:11:43.740 --> 00:11:46.740

And that seems like the right way to be using Python.

00:11:46.740 --> 00:11:48.820

So what's the actual package I use?

00:11:48.860 --> 00:11:51.920

If I were to say pip install something, what do I type to get started?

00:11:51.920 --> 00:11:56.480

So unfortunately, OpenCV is not pip installable.

00:11:56.480 --> 00:11:58.980

I wish it was, but it is not.

00:11:58.980 --> 00:12:04.480

And it is not the easiest package to get installed on your system.

00:12:04.480 --> 00:12:14.280

If you're using Ubuntu or any Debian-based operating system, you technically can do an app git install.

00:12:14.280 --> 00:12:17.680

But that's going to pull down a previous version of OpenCV.

00:12:18.100 --> 00:12:22.180

You're going to run into a lot of problems with the Python bindings, and it's not a very

00:12:22.180 --> 00:12:22.720

good experience.

00:12:22.720 --> 00:12:28.220

So what you actually have to do is compile it from source, download the code from their

00:12:28.220 --> 00:12:32.720

GitHub or the SourceForge account, and manually compile it and install it.

00:12:32.720 --> 00:12:37.980

And in fact, that's really the only way to do it if you're interested in using virtual environments,

00:12:37.980 --> 00:12:44.720

which, as most Python developers are interested in, sequestering their packages.

00:12:45.400 --> 00:12:45.840

Yeah, absolutely.

00:12:45.840 --> 00:12:46.300

Okay.

00:12:46.300 --> 00:12:47.940

So I go and I download that.

00:12:47.940 --> 00:12:51.800

And then what packages are in there that I would work with?

00:12:51.800 --> 00:12:53.700

Is that CV2?

00:12:53.700 --> 00:12:55.080

Is that the one I would import?

00:12:55.080 --> 00:12:55.720

Yep.

00:12:55.780 --> 00:13:01.440

So if you were to open up your favorite editor, you would just type in import CV2, and I'll

00:13:01.440 --> 00:13:04.100

give you access to all of your OpenCV bindings.

00:13:04.100 --> 00:13:04.900

Okay, great.

00:13:04.900 --> 00:13:11.440

And now I looked at some samples on your blog about how I might go and grab like an image

00:13:11.440 --> 00:13:13.780

from a camera hooked to an Adreno.

00:13:14.640 --> 00:13:20.220

Maybe we could talk a little bit about the type of hardware that you need and the spectrum

00:13:20.220 --> 00:13:24.500

of devices you can interact with and that kind of stuff before we get into the more theoretical

00:13:24.500 --> 00:13:24.880

bits.

00:13:24.880 --> 00:13:25.700

Sure.

00:13:25.700 --> 00:13:31.760

So that's kind of the cool thing about OpenCV is that it's meant to be run in real time.

00:13:31.760 --> 00:13:38.800

So you can easily process video files, raw video streams without too much of a problem,

00:13:38.900 --> 00:13:41.780

again, depending on the complexity of your algorithm.

00:13:41.780 --> 00:13:46.800

And OpenCV is meant to run on a variety of different devices.

00:13:46.800 --> 00:13:54.660

I personally develop applications on my MacBook, but I also own a Raspberry Pi and a camera module

00:13:54.660 --> 00:13:55.760

for the Raspberry Pi.

00:13:55.760 --> 00:14:02.960

And using OpenCV, I can access the Raspberry Pi video stream and then actually build like

00:14:02.960 --> 00:14:08.000

a home surveillance system using nothing but OpenCV and a Raspberry Pi.

00:14:08.480 --> 00:14:14.900

I built this one project where I had a Raspberry Pi camera mounted on my kitchen cabinets looking

00:14:14.900 --> 00:14:17.000

over the front door of my apartment.

00:14:17.000 --> 00:14:22.280

And it would detect motion, such as when you're opening the door and somebody's walking inside.

00:14:22.280 --> 00:14:27.980

So once it detected motion, it would snap a photo of whoever was walking inside, try and identify

00:14:27.980 --> 00:14:33.320

their face, and then it would take that screenshot or the screen capture and then upload it to my

00:14:33.320 --> 00:14:34.260

personal Dropbox.

00:14:34.260 --> 00:14:37.340

So I had like this real-time home surveillance system.

00:14:37.940 --> 00:14:39.940

That was really, really cool to develop.

00:14:39.940 --> 00:14:43.140

And again, like this is using simple hardware.

00:14:43.140 --> 00:14:48.480

The Raspberry Pi is not a powerful machine, but you could still build some really cool computer

00:14:48.480 --> 00:14:49.760

vision applications with it.

00:14:49.760 --> 00:14:50.420

Yeah.

00:14:50.420 --> 00:14:51.680

And it's cheap too, right?

00:14:51.680 --> 00:14:52.700

Yeah.

00:14:52.700 --> 00:14:58.460

The Pi itself is, I think, $35 and probably another $20 for the camera module.

00:14:58.460 --> 00:14:59.460

Yeah.

00:14:59.460 --> 00:15:01.240

That's really easy to get started.

00:15:01.240 --> 00:15:02.920

So very cool.

00:15:02.920 --> 00:15:03.360

Very cool.

00:15:03.360 --> 00:15:05.620

How does computer vision work?

00:15:05.620 --> 00:15:10.600

I mean, I have a little bit of a background in trying to identify things and images.

00:15:11.280 --> 00:15:18.240

I worked at this place called eye tracking, E-Y-E, tracking, not I, the letter I, tracking.

00:15:18.240 --> 00:15:22.380

And we did a lot of stuff with image recognition and detecting eyes.

00:15:22.380 --> 00:15:26.560

And I know enough to know that it seems really hard, but how does it work?

00:15:27.600 --> 00:15:34.500

So computer vision as a field is really just encompassing methods on acquiring, processing,

00:15:34.500 --> 00:15:38.860

analyzing, and just understanding and interpreting the contents of an image.

00:15:38.860 --> 00:15:41.080

For humans, this is really, really easy.

00:15:41.440 --> 00:15:43.860

We see a picture of a cat, and we know, like, oh, that's a cat.

00:15:43.860 --> 00:15:46.660

And we see a picture of a dog, and we obviously know that's a dog.

00:15:46.660 --> 00:15:48.820

But a computer, it doesn't have a clue.

00:15:48.820 --> 00:15:53.260

It just sees a bunch of pixels, just a big matrix of pixels.

00:15:53.260 --> 00:15:58.080

And the challenging part, as you suggested, is writing code and creating these algorithms

00:15:58.080 --> 00:16:00.660

that can understand the contents of an image.

00:16:00.660 --> 00:16:06.520

You can't open up your Python source file and then write if statements that say,

00:16:06.740 --> 00:16:12.620