
This is probably my favourite paradox because it is a very basic problem in electromagnetism. It was brought to wide attention by W. Shockley (co–inventor of the transistor) and R. P. James in Physical Review Letters in the late 1960s [1]. The problem is also referred to as the Feynman paradox, for a similar problem appears in the second volume of Feynman's lectures on physics [2]. The solution to the problem appears in [3–4] (for example), but continues to be debated right up to the present day [5–6]. In the past I've made embarassing mistakes because I didn't understand this, so I often return to it.
The basic problem can be understood through considering the situation illustrated in the figure below.
An electric charge is at rest outside a neutral body (a magnet, also at rest), through which there flows an electrical current, . The charge, of mass and charge , at position is subject to zero Lorentz force,
(1) 
because the magnet produces no electric field and the charge is at rest. The centre of mass of the magnet, is also subject to the Lorentz force,
(2) 
where is the charge density, and is the total mass of the magnet. The right hand side of (2) is zero because , and the stationary charge produces no magnetic field. Moreover, the interaction of the current density with its own magnetic field does not produce any motion of the magnet's centre of mass.
So initially all is well, the charge and magnet are both at rest (and therefore so is the centre of mass of the total system) and there is nothing to disturb this situation. However, suppose we now act to reduce to zero. This could happen in a number of ways. For example, the current could be due to two closely spaced counterrotating, oppositely charged plates and these could be allowed to touch such that friction brings them to rest. Alternatively the current could be flowing through a superconductor and we could remove the cooling mechanism so that the resistance increases to the point where the current is eliminated. The process can take an arbitrarily long time, so we can ignore any radiation produced by the change in current. However it is done, the change in current will come with a corresponding change in magnetic flux, and therefore a nonzero electric field at the position of the charge, which can be inferred from Lenz's law,
(3) 
This electric field will cause the charge to accelerate.
Yet there is no equal opposing force on the magnet. One might think that because the charge is set into motion, this will generate a magnetic field that will push the magnet. There is such a force, but it is a factor of smaller than the force on the charge, and depends on . To simplify matters we consider a heavy charge, which is subject to the same force, but then moves with a tiny velocity in comparison to the speed of light, and thus produces negligible magnetic field. We then have a paradox: after the current, has been reduced to zero, the centre of mass of the total system has been set into motion. In Newtonian physics, the centre of mass of a closed system moves at constant velocity as a consequence of Newton's third law. Does the interaction of charges and magnets not obey Newton's third law, even in the quasistatic limit?
[1]  Shockley, W. and James, R. P. "Try Simplest Cases Discovery of Hidden Momentum Forces on Magnetic Currents" Phys. Rev. Lett. 18 876 (1967) 
[2]  Feynman, R. P. "The Feynman Lectures on Physics" Vol. II, Addison Wesley (1964) 
[3]  Coleman, S. and Van Vleck, J. H. "Origin of Hidden Momentum Forces on Magnets" Phys. Rev. 171 1370 (1968) 
[4]  Furry, W. H. "Examples of Momentum Distributions in the Electromagnetic Field and in Matter", Am. J. Phys. 37 621 (1969) 
[5]  Boyer, T. H. "Concerning hidden momentum", Am. J. Phys. 76 190 (2008) 
[6]  Babson, D., Reynolds, S. P., Bjorkquist, R. and Griffiths, D. J. "Hidden momentum, field momentum, and electromagnetic impulse", Am. J. Phys. 77 826 (2009) 