Image Formation – University of Toronto

 

 

Image Formation

Goal:

Introduce the elements of camera models, optics, and image

formation.

Motivation: Camera models, together with radiometry and reflectance

models, allow us to formulate the dependence of image color/intensity

on material reflectance, surface geometry, and lighting.

• Based on these models, we can formulate the inference of scene

properties such as surface shape, reflectance, and scene lighting,

from image data.

• Here we consider the “forward” model. That is, assuming vari-

ous scene and camera properties, what should we observe in an

image?

and Ponce.

Readings: Part I (Image Formation and Image Models) of Forsyth

Matlab Tutorials: colourTutorial.m (in UTVis)

2503: Image Formation

c(cid:13)A.D. Jepson and D.J. Fleet

Page: 1

Camera Elements

image _x000D_
plane

lens

aperture

surface_x000D_
point

optical axis

enclosure

Albrecht Durer (1471-1528):

2503: Image Formation

Page: 2

Thin Lens

Suppose we neglect the thickness of the lens and assume the medium

(e.g. air) is the same on both sides of the lens:

O

Aperture

Nodal Distance( )N D

P, N

F ′

F

Z

f

f

Z ′

Image
Plane

O′

A world point ~O is focussed at ~O′, at the intersection of three rays:

• A ray from ~O through N (the nodal point, or center of projection)

• Two rays, parallel to the optical axis, one in front of and one

behind the lens (aka principal plane), which pass through the
rear/front focal points (F ′ and F ) on the opposite side of the lens.

Remarks:

• Lens Aperture is modeled as an occluder (in the principal plane)
• The point ~O is not in focus on the image plane, but rather at ~O′.

• Thin lens model:

1
Z
• The focal length f is the distance of the point behind the lens, F ′,

1
Z′

1
f

=

+

to which rays of light from infinity converge.

2503: Image Formation

Page: 3

General Lens Model

Given a general lens, a point at ~O is imaged to ~O′, where the locations of ~O and ~O′ are given by the
lens forumla:

z′
y′

Here F , F ′ are focal points, and f , f ′ are focal lengths.

f ′
y !

f
z

~O′ ≡

!

=

,

~O ≡

z
y !

=

f ′
z′

f
y′

.

!

O

y

F

z

Object Space

Lens

y’

z’

F’

O’

Image Space

In general the ratio of focal lengths equals the ratio of the indices of refraction of the pre- and post-
lens material, that is f /f ′ = n/n′ (eg. f 6= f ′ for eyes, but f = f ′ for most cameras). The index of
refraction of a material is the ratio of the speed of light in a vacuum over the speed of light in that

As for a thin lens, the formation of the image of ~O can be interpreted geometrically as the intersec-
tion of three canonical rays, which are determined by the cardinal points of the lens. The cardinal

medium.

points are:

Focal Points F, F ′ provide origins for the object and image spaces.

Nodal Points N, N ′, are defined using the lens axis, F, F ′, and focal lengths, f, f ′.

Principal Points P, P ′ are also defined using the lens axis, F, F ′, and focal lengths, f, f ′.

O

f’

f

PN

P’

F’

F

N’

O’

f’

f

2503: Image Formation

Notes: 4

An alternative coordinate system which is sometimes used to write the lens formula is to place the

origins of the coordinates in the object and image space at the principal points P and P’, and flip

both the z-axes to be pointing away from the lens. These new z-coordinates are:

Solving for z and z′ and substituting into the previous lens formula, we obtain:

Lens Formula

ˆz = f − z,
ˆz′ = f ′ − z′.

(f ′ − ˆz) = f f ′/(f − ˆz),

f f ′ = (f ′ − ˆz′)(f − ˆz)
ˆz′ ˆz = ˆz′f + ˆzf ′
f
ˆz

f ′
ˆz′

1 =

+

The last line above is also known as the lens formula. As we have seen, it is equivalent to the one

on the previous page, only with a change in the definition of the coordinates.

For cameras with air both in front of and behind the lens, we have f = f’. This simplifies the lens

formula above. Moreover, the nodal and principal points coincide in both the object and scene spaces
(i.e., N = P and N ′ = P ′ in the previous figure).

Finally it is worth noting that, in terms of image formation, the difference between this general lens

model and the thin lens approximation is only in the displacement of the cardinal points along the

optical axis. That is, effectively, the change in the imaging geometry from a thin lens model to

the general lens model is simply the introduction of an absolute displacement in the image space

coordinates. For the purpose of modelling the image for a given scene, we can safely ignore this

displacement and use the thin lens model. When we talk about the center of projection of a camera

in a world coordinate frame, however, it should be understood we are talking about the location of

the nodal point N in the object space (and not N’ in the image space). Similarly, when we talk about

the nodal distance to the image plane, we mean the distance from N’ to the image plane.

2503: Image Formation

Notes: 5

Image of a Lambertian Surface

!xp

!A

!xI

N D

image_x000D_
plane

!V

dAp

!Ns

!L

!xs

The irradiance on the image plane is

I(λ, ~xI) = Tl

r(λ) I(λ, ~xs)

dΩp dAV
dAI

Here

• ~NI is normal to the image plane, and ~A is the optical axis;

• Tl ∈ (0, 1] is the transmittance of the lens;

• dAI is the area of each pixel;

• dAp is the area of the aperture;

• dΩp is the solid angle of the aperture from the surface point ~xs;

• dAV is the cross-sectional area, perpendicular to the viewing di-
rection, of the portion of the surface imaged to the pixel at ~xI.

2503: Image Formation

Page: 6

Derivation of the Image of a Lambertian Surface

From our notes on Lambertian reflection, the radiance (spectral density) of the surface is

R(λ, ~xs; ~V ) = r(λ) I(λ, ~xs; ~Ns) = r(λ) ⌊ ~N · ~L⌋ I(λ, ~xs; ~L) .

The reflected radiance is measured in Watts per unit wavelength, per unit cross-sectional area per-

pendicular to the viewer, per unit steradian.

The total power (per unit wavelength) from the patch dAV , arriving on the aperature, is therefore

P (λ) = R(λ, ~xs; ~V )dΩpdAV .

A fraction Tl of this is transmitted through the lens, and ends up on a pixel of area dAI. Therefore,
the pixel irradiance spectral density is

I(λ, ~xI, ~nI) =

Tl P (λ)
dAI

,

which is the expression on the previous page.

To simplify this, first compute the solid angle of the lens aperature, with respect to the surface
point ~xs. Given the area of the aperature, dAp, and the optical axis, ~A, which is assumed to be
perpendicular to the aperture, we have

dΩp =

|~V · ~A| dAp
||~xp − ~xs||2 .

Here the numerator is the cross-sectional area of the aperature viewed from the direction ~V . The
denominator scales this foreshortened patch back to the unit sphere to provide the desired measure of
solid angle. Secondly, we need the foreshortened surface area dAV which projects to the individual
pixel at ~xI having area dAI. These two patches are related by rays passing through the center of
projection ~xp; they have the same solid angle with respect to ~xp. As a result,

The distance in the denominator here can be replaced by

dAV = ||~xp − ~xs||2 |~V · ~A| dAI
||~xp − ~xI||2

||~xp − ~xI|| =

ND
|~V · ~A|

.

Substituting these expressions for dΩp, dAV , and ||~xp − ~xI|| gives the equation for the image irradi-
ance due to a Lambertian surface on the following page.

2503: Image Formation

Notes: 7

Image of a Lambertian Surface (cont.)

This expression for the irradiance due to a Lambertian surface sim-

plifies to

I(λ, ~xI; ~NI) = Tl

dAp
|N D|2 | ~A · ~V |4 r(λ) ⌊ ~N · ~L⌋ I(λ, ~xs; ~L)

where dAp is the area of the aperture.

Note that image irradiance

• does not depend on the distance to the surface ||~xs − ~xp||

(as the distance to the surface increases, the surface area ”seen”

by a pixel also increases to compensate for the distance change);

• falls off like cos(θ)4 in the corners of the image where θ is the
angle between the viewing direction ~V and the optical axis ~A.
For wide angle images, there is a significant roll-off in the image

intensity towards the corners.

The fall off of the brightness in the corners of the image is called

vignetting. The actual vignetting obtained depends on the internal
structure of the lens, and will deviate from the above cos(θ)4 term.

2503: Image Formation

Page: 8