New to diesel - strange noises

is it only when the engine is cold?
how many miles are on it?

What was done on the last service?
When was the timing belt last done?
If you could record the noise and post it somewhere with a link it'd help.

But it sounds like it might be worn tappets or something similar but its liable that it's just diesel knock if it's only a problem when cold.

If it's diesel knock these are the main reasons:

poor fuel - use good diesel (eg. vPower) or add some fuel additive every time you fill up (eg. millers). I'd recommend this anyway as it makes the engine run smoother anyway.

blocked intake - Due to the EGR valve pumping soot into the intake and the CCV blowing hot oil vapour into the intake system, the intake manifold can get blocked.
Remove and clean the intake manifold and egr valve. This is heavily documented on TDIClub and some members here say that oven cleaner does the best job at lifting the carbon.
I'd definitely recommend doing this as with that mileage there's bound to be heavy buildup.

faulty MAF/MAP sensor - These two sensors send the ECU information about the amount of air it's getting. The ECU adjusts fueling accordingly so if one of them is dodgy the engine might be getting the wrong amount of fuel and shuddering/misfiring (especially when cold).
The MAF sensor is infamous for going south on VAG TDI's and sometimes it does not log a fault although it usually causes power problems.

And as for the mileage it's easily got another 70k in it. (they'll go till well over 250k if well maintained.)

good to hear, I'd still recommend using millers/other fuel additive and giving the intake a clean out though.

whilst they are doing the cambelt, do the water pump as well.
need to remove cambelt to do water pump so easier in one job.

make sure as well the water pump has metal impellars

I wouldn't let it run out if I was you...

The car will more than likely make a loud high pitched beep when your tank gets low anyway (scared the life out of me the first time i heard it).

Any time I do the fuel filter I just keep cranking it until fuel gets to the block but I've never ran all the fuel out of the lines so not 100% sure.

And during the winter it's a good idea to keep the tank full as this prevents condensation buildup in the tank, thus preventing you from having to bleed the fuel filter as often. (yes i am that lazy)

Back when I had my Mk 4 Ibiza Sport (PD130), as time and miles increased, I too noticed a similar noise, and I think I did once describe it in exactly the same way - like someone had chucked stones in a cement mixer or put pebbles in the gearbox.

Car always ran fine for many miles, never did know the cause.

I assumed that it was backlash in the gearbox when the engine was idling (have you seen how much the engines rock to'n'fro on their mounts at idle?), perhaps because the DMF was past it's best.

But if it stopped when your PS belt failed, then can't really argue with that.
Must have been a different fault on my Ibiza, as that had electric power steering, so no belt.
Have you got your car back, and has the noise gone ?
Just wondering if it was the belt, or maybe a dodgy PS pump that's causing the noise (hence why it stopped when the belt failed). Could also have been a dodgy pump that caused the belt to go in the first place....