Your body naturally produces cholesterol, and it will compensate by tuning the level of production up or down depending on your current serum cholesterol level.
Also, eat the egg yolks. There are some recent studies that show that the cholesterol in eggs is actually not absorbed by your body. Instead, it binds with a choline molecule also found in egg yolks. This explains the relationship that eggs have little to no impact on your serum cholesterol levels.