Image Formation
Goal:
Introduce the elements of camera models, optics, and image
formation.
Motivation: Camera models, together with radiometry and reflectance
models, allow us to formulate the dependence of image color/intensity
on material reflectance, surface geometry, and lighting.
• Based on these models, we can formulate the inference of scene
properties such as surface shape, reflectance, and scene lighting,
from image data.
• Here we consider the “forward” model. That is, assuming vari-
ous scene and camera properties, what should we observe in an
image?
and Ponce.
Readings: Part I (Image Formation and Image Models) of Forsyth
Matlab Tutorials: colourTutorial.m (in UTVis)
2503: Image Formation
c(cid:13)A.D. Jepson and D.J. Fleet
Page: 1
Camera Elements
image _x000D_
plane
lens
aperture
surface_x000D_
point
optical axis
enclosure
Albrecht Durer (1471-1528):
2503: Image Formation
Page: 2
Thin Lens
Suppose we neglect the thickness of the lens and assume the medium
(e.g. air) is the same on both sides of the lens:
O
Aperture
Nodal Distance( )N D
P, N
F ′
F
Z
f
f
Z ′
Image
Plane
O′
A world point ~O is focussed at ~O′, at the intersection of three rays:
• A ray from ~O through N (the nodal point, or center of projection)
• Two rays, parallel to the optical axis, one in front of and one
behind the lens (aka principal plane), which pass through the
rear/front focal points (F ′ and F ) on the opposite side of the lens.
Remarks:
• Lens Aperture is modeled as an occluder (in the principal plane)
• The point ~O is not in focus on the image plane, but rather at ~O′.
• Thin lens model:
1
Z
• The focal length f is the distance of the point behind the lens, F ′,
1
Z′
1
f
=
+
to which rays of light from infinity converge.
2503: Image Formation
Page: 3
General Lens Model
Given a general lens, a point at ~O is imaged to ~O′, where the locations of ~O and ~O′ are given by the
lens forumla:
z′
y′
Here F , F ′ are focal points, and f , f ′ are focal lengths.
f ′
y !
f
z
~O′ ≡
!
=
,
~O ≡
z
y !
=
f ′
z′
f
y′
.
!
O
y
F
z
Object Space
Lens
y’
z’
F’
O’
Image Space
In general the ratio of focal lengths equals the ratio of the indices of refraction of the pre- and post-
lens material, that is f /f ′ = n/n′ (eg. f 6= f ′ for eyes, but f = f ′ for most cameras). The index of
refraction of a material is the ratio of the speed of light in a vacuum over the speed of light in that
As for a thin lens, the formation of the image of ~O can be interpreted geometrically as the intersec-
tion of three canonical rays, which are determined by the cardinal points of the lens. The cardinal
medium.
points are:
Focal Points F, F ′ provide origins for the object and image spaces.
Nodal Points N, N ′, are defined using the lens axis, F, F ′, and focal lengths, f, f ′.
Principal Points P, P ′ are also defined using the lens axis, F, F ′, and focal lengths, f, f ′.
O
f’
f
PN
P’
F’
F
N’
O’
f’
f
2503: Image Formation
Notes: 4
An alternative coordinate system which is sometimes used to write the lens formula is to place the
origins of the coordinates in the object and image space at the principal points P and P’, and flip
both the z-axes to be pointing away from the lens. These new z-coordinates are:
Solving for z and z′ and substituting into the previous lens formula, we obtain:
Lens Formula
ˆz = f − z,
ˆz′ = f ′ − z′.
(f ′ − ˆz) = f f ′/(f − ˆz),
f f ′ = (f ′ − ˆz′)(f − ˆz)
ˆz′ ˆz = ˆz′f + ˆzf ′
f
ˆz
f ′
ˆz′
1 =
+
The last line above is also known as the lens formula. As we have seen, it is equivalent to the one
on the previous page, only with a change in the definition of the coordinates.
For cameras with air both in front of and behind the lens, we have f = f’. This simplifies the lens
formula above. Moreover, the nodal and principal points coincide in both the object and scene spaces
(i.e., N = P and N ′ = P ′ in the previous figure).
Finally it is worth noting that, in terms of image formation, the difference between this general lens
model and the thin lens approximation is only in the displacement of the cardinal points along the
optical axis. That is, effectively, the change in the imaging geometry from a thin lens model to
the general lens model is simply the introduction of an absolute displacement in the image space
coordinates. For the purpose of modelling the image for a given scene, we can safely ignore this
displacement and use the thin lens model. When we talk about the center of projection of a camera
in a world coordinate frame, however, it should be understood we are talking about the location of
the nodal point N in the object space (and not N’ in the image space). Similarly, when we talk about
the nodal distance to the image plane, we mean the distance from N’ to the image plane.
2503: Image Formation
Notes: 5
Image of a Lambertian Surface
!xp
!A
!xI
N D
image_x000D_
plane
!V
dAp
!Ns
!L
!xs
The irradiance on the image plane is
I(λ, ~xI) = Tl
r(λ) I(λ, ~xs)
dΩp dAV
dAI
Here
• ~NI is normal to the image plane, and ~A is the optical axis;
• Tl ∈ (0, 1] is the transmittance of the lens;
• dAI is the area of each pixel;
• dAp is the area of the aperture;
• dΩp is the solid angle of the aperture from the surface point ~xs;
• dAV is the cross-sectional area, perpendicular to the viewing di-
rection, of the portion of the surface imaged to the pixel at ~xI.
2503: Image Formation
Page: 6
Derivation of the Image of a Lambertian Surface
From our notes on Lambertian reflection, the radiance (spectral density) of the surface is
R(λ, ~xs; ~V ) = r(λ) I(λ, ~xs; ~Ns) = r(λ) ⌊ ~N · ~L⌋ I(λ, ~xs; ~L) .
The reflected radiance is measured in Watts per unit wavelength, per unit cross-sectional area per-
pendicular to the viewer, per unit steradian.
The total power (per unit wavelength) from the patch dAV , arriving on the aperature, is therefore
P (λ) = R(λ, ~xs; ~V )dΩpdAV .
A fraction Tl of this is transmitted through the lens, and ends up on a pixel of area dAI. Therefore,
the pixel irradiance spectral density is
I(λ, ~xI, ~nI) =
Tl P (λ)
dAI
,
which is the expression on the previous page.
To simplify this, first compute the solid angle of the lens aperature, with respect to the surface
point ~xs. Given the area of the aperature, dAp, and the optical axis, ~A, which is assumed to be
perpendicular to the aperture, we have
dΩp =
|~V · ~A| dAp
||~xp − ~xs||2 .
Here the numerator is the cross-sectional area of the aperature viewed from the direction ~V . The
denominator scales this foreshortened patch back to the unit sphere to provide the desired measure of
solid angle. Secondly, we need the foreshortened surface area dAV which projects to the individual
pixel at ~xI having area dAI. These two patches are related by rays passing through the center of
projection ~xp; they have the same solid angle with respect to ~xp. As a result,
The distance in the denominator here can be replaced by
dAV = ||~xp − ~xs||2 |~V · ~A| dAI
||~xp − ~xI||2
||~xp − ~xI|| =
ND
|~V · ~A|
.
Substituting these expressions for dΩp, dAV , and ||~xp − ~xI|| gives the equation for the image irradi-
ance due to a Lambertian surface on the following page.
2503: Image Formation
Notes: 7
Image of a Lambertian Surface (cont.)
This expression for the irradiance due to a Lambertian surface sim-
plifies to
I(λ, ~xI; ~NI) = Tl
dAp
|N D|2 | ~A · ~V |4 r(λ) ⌊ ~N · ~L⌋ I(λ, ~xs; ~L)
where dAp is the area of the aperture.
Note that image irradiance
• does not depend on the distance to the surface ||~xs − ~xp||
(as the distance to the surface increases, the surface area ”seen”
by a pixel also increases to compensate for the distance change);
• falls off like cos(θ)4 in the corners of the image where θ is the
angle between the viewing direction ~V and the optical axis ~A.
For wide angle images, there is a significant roll-off in the image
intensity towards the corners.
The fall off of the brightness in the corners of the image is called
vignetting. The actual vignetting obtained depends on the internal
structure of the lens, and will deviate from the above cos(θ)4 term.
2503: Image Formation
Page: 8