The Mystery of Vacuum Energy

Why should the vacuum have any energy? The vacuum is the state of lowest energy of a physical theory. Particles are excitations of the vacuum, like guitar music is excitations of

guitar strings. The vacuum corresponds to silence. But the nature of the vacuum (the unplayed guitar) determines the particle spectrum (the possible sounds which can be produced with the given guitar).

For all physical theories except gravity, the absolute value which we give to the state of lowest energy, the vacuum, is of no importance, unmeasurable. Only differences can be measured.

Like in classical physics: whether you consider a simple quadratic potential (Figure, left) or a quadratic potential shifted by a constant (Figure, right) makes absolutely no difference. The only relevant quantities are the initial position and velocity of the particle. The former determines the potential energy up to the irrelevant constant.

The same applies in quantum field theory: the value of the vacuum energy has no effect on the interactions of particles; we cannot even calculate it, it is undetermined. Only differences of vacuum energies can be calculated and measured.

For example: If two metallic plates are positioned at a distance D from each other in otherwise empty space, the vacuum of the electromagnetic field changes. The boundary conditions require e.g. that the tangential electric field at the position of the plates vanish. Only photons (excitations of the electromagnetic field) with wavelength which are such that D is a multiple of it are admitted. The energy of a photon is inversely proportional to its wavelength, hence only photons with certain energies can exist between the two plates.

If the distance of the plates is changed, the possible photon energies and also the vacuum energy change. This leads to an attractive force between the plates. This “Casimir effect”¹ has been measured with great accuracy², proving at its best the physical reality and non-triviality of the quantum vacuum.

Also during phase transitions (e.g. the freezing of water into ice) the possible elementary excitations of the fields and therefore the quantum vacuum energy change in a well defined calculable way. Furthermore, the presence of a positively charged atomic nucleus changes the vacuum energy and thereby affects the atomic energy spectra. For the hydrogen atom this vacuum contribution has first been measured by Lamb³ (“Lamb shift”) in perfect agreement with the calculated value.

Changes of the vacuum energy produce measurable physical effects. However, its absolute value affects only the gravitational field. The gravitational field which defines the metric and curvature of spacetime is determined by the sum of all forms of energy and momentum. By symmetry reasons, the so called energy momentum tensor of the vacuum must be of a form such that its pressure is exactly the negative of its energy density. If the energy density of the vacuum is positive, its pressure is thus necessarily negative.

In 1916, when Einstein was searching for a static and homogeneous solution of his field equations, he discovered that one can add to them a term proportional to the so called cosmological constant ë, which then allows for static solutions without influencing the local gravitational field e.g. in the solar system. Later, when the expansion of the Universe was discovered, Einstein discarded this “cosmological term”.

The cosmological term is exactly equivalent to a vacuum energy. Vacuum energy and the cosmological constant cannot be distinguished by any experiment and are therefore physically equivalent.

This led me, and many other physicists to the conviction that the freedom to add a cosmological constant corresponds to the freedom to choose the absolute scale of the vacuum energy and both of them should be chosen so that they cancel each other and have no net observational consequence, also gravitationally. We hoped, that a consistent theory of quantum gravity, once found, would tell us how to do this in detail.

However, the observed accelerated expansion of the Universe⁵ indicates the presence of vacuum energy (or some other form of energy with substantial negative pressure, see contribution by Luca Amendola) corresponding to an energy of about 10⁴eV per centimeter cubed. (Here, 1eV is the energy gained by an electron when accelerated over a voltage difference of 1Volt.) For ordinary matter, gravity is an attractive force which decelerates the expansion of the Universe. But the true source of gravity is not just the mass-energy density but actually the energy density plus three times the pressure. Therefore, a component with negative pressure can lead to repulsive gravity and so accelerate the expansion of the Universe.

It seems hopeless to explain this tiny but non-vanishing energy as the difference of the vacuum energies from a phase transition. For all known cosmological phase transitions, the differences are much larger; and if they play a role in cosmology, we would expect the largest one to dominate. But its value should be of the order of the Planck scale, about 10¹²⁵eV per centimeter cubed, which is more than 120 orders of magnitude larger than the measured value!

Furthermore, the vacuum energy density is just of the same order as the energy density of matter in the Universe today. The latter will be much smaller at all later cosmological times and it was much larger at all previous times. A very curious accident indeed?

It is clear that we seriously misunderstand something here...

References

[1] H.B.G. Casimir, Proc. K. Ned. Akad. Wet. 51, 793 (1948).
[2] U. Mohideen and A. Roy, Phys. Rev. Lett. 81, 4549 (1998).
[3] W.E. Lamb, Jr., Rep. Progr. Phys. 14, 19 (1951).
[4] S. Perlmutter et al., Astrophys. J. 517, 565 (1999);
    A. Riess et al., Astrophys. J. 607, 665 (2004), and references therein.