Description

EECS16A Imaging 2
Agenda
● Quick overview + review
● Images as matrices and vectors
● Pixel-by-pixel scanning
● Reconstructing scans as images
● Lab-specific simulation directions
Last Week: Imaging 1
Today’s Lab: Single Pixel Scanning
● Circuit from last week measures light intensity
● Simulated projector illuminates image in a controlled way
● Python programming to reconstruct image
Why?
● Imaging 1:
○ Finding a link between physical quantities and voltage is powerful
○ If you can digitize it, you can do anything
(IOT devices, internet, code, processing)
● Imaging 2:
○ What measurements are good measurements?
■ Remember Kody and Nara from Dis0C?
Kody and Nara

Illuminating the Big Picture
● Linear dependence
○ When can you recover your image?
○ Does it matter what mask matrix you pick?
○ Does it matter how you cover the pixels? ● Invertibility
○ When can you solve Ax = b?
○ How does this relate to our system?
○ How does this affect the way we pick our masking matrix?
Real-life Setup
Real-life Setup (cont.)
1. Draw a “simple” image
2. Project masks (rows of H) onto it in a dark environment
3. Measure ambient light sensor reading s
4. Multiply by H inverse to find i (= H-1.s)
Sample Real-Life Setup Images

This Lab: Software Simulator Setup
1. Upload (“simple”) image of an object (or use default)
2. Shrink your image (preferably 32×32)
3. ‘Project’ masks (rows of H) onto it and
“measure” s using matrix multiplication
4. Multiply by H inverse to find i (= H-1s)
● What are the unknowns in our system?
● Want to do an experiment to get information about these
unknowns
● We can do a lot of interesting processing on vectors, but we need to convert the image into one first
○ In lecture and discussion, you have seen how to turn an image into a vector.
How?

[0] [1]
[2] [3]
[4] [5]

[0] [1]
[2] [3]
[4] [5]

[0]
[1]

[0] [1]
[2] [3]
[4] [5]

[0]
[1]
[2]
[3]

[0] [1]
[2] [3]
[4] [5]

[0]
[1]
[2]
[3]
[4]
[5]

[0] [1]
[2] [3]
[4] [5]

[0]
[1]
[2]
[3]
[4]
[5]

Question Time
● To read all the pixels of a 4×4 image, how many pixel-by-pixel scans do we need to do?

1, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
Representing our Masks in Python
Imaging Mask 0
0, 1, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
Representing our Masks in Python

Turning the Masks Into Vectors
0, 1, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0
Turning the Masks Into Vectors

1, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
1
0
0
0
0
1, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
⁞
1
0
0
0
0
1, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
1 0 0 0 0 0 0 0 …
0
0 ⁞

0, 1, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0

1 0 0 0 0 0 0 0 …

0
1
0
0
0
0, 1, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
1 0 0 0 0 0 0 0 …
0
0
1
0
0
0
0, 1, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
1 0 0 0 0 0 0 0 …
0 1 0 0 0 0 0 0 …
0
⁞ ⁞

1 0 0 0 0 0 0 0 …
0 1 0 0 0 0 0 0 …
0 0 1 0 0 0 0 0 …

1 0 0 0 0 0 0 0 …
0 1 0 0 0 0 0 0 …
0 0 1 0 0 0 0 0 …
0 0 0 1 0 0 0 0 …

Each Row is a Different Experiment

1 0 0 0 0 0 0 0 …
0 1 0 0 0 0 0 0 …
0 0 1 0 0 0 0 0 …
0 0 0 1 0 0 0 0 …
0 0 0 0 1 0 0 0 …
0 0 0 0 0 1 0 0 …
0 0 0 0 0 0 1 0 …
…

Measuring a Pixel is Matrix-Vector Multiplication
i1
i2
i3

in
1 0 0 0 0 0 0 0 …
0 1 0 0 0 0 0 0 …
0 0 1 0 0 0 0 0 …
0 0 0 1 0 0 0 0 …
0 0 0 0 1 0 0 0 …
0 0 0 0 0 1 0 0 …
0 0 0 0 0 0 1 0 …
…
s1
s2
s3

sn
=
Masking Matrix H Unknown, Recorded vectorized Sensor image, readings,
Measuring a Pixel is Matrix-Vector Multiplication

● We know H and we have the sensor readings, how do we get the image?
● How do we solve this?
● When can we solve this?
○ Conditions on H

1 0 0 0 0 0 0 0 …
0 1 0 0 0 0 0 0 …
0 0 1 0 0 0 0 0 …
0 0 0 1 0 0 0 0 …
0 0 0 0 1 0 0 0 …
0 0 0 0 0 1 0 0 …
0 0 0 0 0 0 1 0 …
…

1 0 0 0 0 0 0 0 …
0 1 0 0 0 0 0 0 …
0 0 1 0 0 0 0 0 …
0 0 0 1 0 0 0 0 …
0 0 0 0 1 0 0 0 …
0 0 0 0 0 1 0 0 …
0 0 0 0 0 0 1 0 …
…
0
1
0
0
0
1 0 0 0 0 0 0 0 …
0 1 0 0 0 0 0 0 …
0 0 1 0 0 0 0 0 …
0 0 0 1 0 0 0 0 …
0 0 0 0 1 0 0 0 …
0 0 0 0 0 1 0 0 …
0 0 0 0 0 0 1 0 …
…
0
0
1
0
0
0
0, 1, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
1 0 0 0 0 0 0 0 …
0 1 0 0 0 0 0 0 …
0 0 1 0 0 0 0 0 …
0 0 0 1 0 0 0 0 …
0 0 0 0 1 0 0 0 …
0 0 0 0 0 1 0 0 …
0 0 0 0 0 0 1 0 …
…
0
0
1
0
0
0
0, 1, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0
1 0 0 0 0 0 0 0 …
0 1 0 0 0 0 0 0 …
0 0 1 0 0 0 0 0 …
0 0 0 1 0 0 0 0 …
0 0 0 0 1 0 0 0 …
0 0 0 0 0 1 0 0 …
0 0 0 0 0 0 1 0 …
…
0
⁞ ⁞ ⁞

What Makes a Mask Good?
● Linearly independent columns →
Invertible
○ Can’t get a solution without this
○ There is a unique solution
● What would be a bad mask?
● Food for thought: Are all invertible matrices equally as good?
○ Find out in Imaging 3 later this week
Cumulative Scanning
Important Notes
1. Pick a simple image! Quality can be lost when resizing.
2. Use a short / simple imagePath name
a. you’ll have to fill this in for some cells
b. default is the home directory for the lab
3. Before starting the simulations, open up the link (in the directions above the code cell) to the display view in a different tab & observe as the cell is running
4. Read simulation descriptions carefully!
a. you might have to manually set height & width because the defaults are 32×32
5. Each mask section includes ideal + noisy imaging
a. noisy imaging meant to simulate real projector behavior
b. don’t worry about this for now — more to come in Img 3