Topic: GoldenEye 007/Perfect Dark HD Music (48kHz/32-bit) (Read 33499 times)

L. Spiro · « **on:** February 04, 2022, 03:40:05 pm »

There were always minor (or major) issues in my previous HD restores of the GoldenEye 007 OST.

I’ve spent the past 2 months working day-in and day-out on a tool I call Nintendo Synthy-4 whose job is to render Nintendo 64 music the way Nintendo 64 did, but with high-resolution versions of all of its routines.
I’ve been debugging the games and copying every routine exactly, taking care of every little detail. For example, when you change a track’s volume to 0, it takes 4 samples to reach 0 volume.
And here’s the exact vibrato routine (except in 64-bit instead of 32-bit):

Code: [Select]

		/**
		 * Gets the vibrato depth given the game's 0-255 value.
		 *
		 * \param _ui8Val The value to convert.
		 * \return Returns the converted vibrato depth.
		 */
		static double					VibratoDepthToReal( uint8_t _ui8Val ) {
			return std::pow( 1.030992984771728515625f, _ui8Val );
		}

Code: [Select]

			void						Set( uint8_t _ui8Rate, uint8_t _ui8Depth, uint8_t _ui8Delay, uint32_t _uiMasterSamplingRate ) {
				dDepth = VibratoDepthToReal( _ui8Depth );

				tbDelay.SetRate( ((1.0 / 30.0 / 2.0) * _ui8Delay) * _uiMasterSamplingRate );
				tbRate.SetRate( ((1.0 / 30.0 / 2.0) * (259 - _ui8Rate)) * _uiMasterSamplingRate );
			}

Code: [Select]

			/**
			 * Gets the current vibrato depth.
			 *
			 * \return Returns the current vibrato depth.
			 */
			double						Value() const {
				if ( dDepth ) {
					double dTmp = std::sin( tbRate.Time() * NS4_TWO_PI ) * dDepth;
					// The cast is only a performance hack by the game, so removing it could be an “as-intended” routine with full precision.
					double dRate = std::pow( 1.00057780742645263671875f, static_cast<int32_t>(dTmp / 2.0) ); 
					return dRate;
				}
				return 1.0;
			}

Nintendo Synthy-4 has a huge focus on accuracy, and that goes for the timing as well. I’ve taken measures to ensure it is always sample-accurate and never drifts.
The real game will trigger all of the notes on a tick starting on the same sample, but the time between ticks can vary.
Nintendo Synthy-4 is an offline renderer, so it can take as much time as it needs to render the perfect result. Every note starts on exactly the correct sample, no matter how long the recording is.
It does not accumulate drift either.
For a computer, (1/7.0)+(1/7.0)+(1/7.0)+(1/7.0)+(1/7.0)+(1/7.0)+(1/7.0) == 0.9999999999999997779553950749686919152736663818359375.
But Nintendo Synthy-4 handles all timers like this: 1/7, 2/7, 3/7, 4/7, 5/7, 6/7, 7/7, etc. This means that tiny inaccuracies are not accumulated. Every single sample in the final result is as accurate as every other sample.

I’ve had to use approximations for a lot of things before, but now even the reverb is accurate, with one difference.
This is the actual game’s reverb taps:

Notice it drifts away from 0 toward the end. That is undesirable so I filtered that back into a nice clean reverb. I also eliminated the artifacts that cause major buzzing.

So what you get is the exact reverb but filtered to sound nice and clean.

So this is a truly accurate HD restore of GoldenEye 007 (48 KHz/32-bit):
https://www.youtube.com/playlist?list=PLWsnao9n727PLCC-QTWwhYhEdqXd5GPce

Samples:
https://www.youtube.com/watch?v=7qacI0HkjrY&list=PLWsnao9n727PLCC-QTWwhYhEdqXd5GPce&index=9
https://www.youtube.com/watch?v=vlrr1SYRXew&list=PLWsnao9n727PLCC-QTWwhYhEdqXd5GPce&index=51

And from Perfect Dark:
https://www.dropbox.com/s/3ia8dwsii0p4d0b/093%20Alien%20Conflict.wav?dl=0
https://www.dropbox.com/s/memh7nhj8moacdj/092%20Maian%20Tears.wav?dl=0
https://www.dropbox.com/s/u93d43skg7ytlvs/065%20Deep%20Sea-%20Nullify%20Threat.wav?dl=0

Enjoy!

L. Spiro

L. Spiro · « **Reply #1 on:** February 05, 2022, 04:41:33 pm »

Oops!

I introduced a reverb bug while refactoring before creating the GoldenEye 007 set.
I’ve updated the WAV pack: https://www.dropbox.com/s/62x3c33yl2oqdg7/GoldenEye%20007%20HDU.rar?dl=0
And only a few videos that were impacted the most by the issue.
My shame is eternal.

L. Spiro

L. Spiro · « **Reply #2 on:** February 21, 2022, 05:03:24 am »

Here is a bit of sampling!
These are 48 KHz/32-bit!
Accuracy Threat: Demon Level
https://www.dropbox.com/s/08bt6sagwi3k8ws/Carrington%20Villa-%20Hostage%20One.wav?dl=0
https://www.dropbox.com/s/r0yxu50ain7s740/Carrington%20Villa-%20Hostage%20One%20Intro-%20Negotiate%20This%21%20%28Secret%20Agent%29.wav?dl=0
As they say: Why approximate when you can exactimate?

No aspect of these are approximate! All algorithms that are applied to notes (note velocity, envelopes, vibrato, pitch-bend, tremolo, stereo-effect, etc.) are replicated using 64-bit fully-detailed implementations! Studio fade-out with LPF down to 100 Hz and normalization to -0.1 dB all done internally in 64-bit double precision before being converted to PCM using nearest-neighbor rather than truncation!

Even the stepping pattern in the vibrato is accurate (top: Game; bottom: Nintendo Synthy-4)!

The reverb? Yep! Exact!

These ain't yo mamma's HD restores!
Woo, what was in those brownies?

Here are some more samples:
https://www.youtube.com/watch?v=TqK4CnDnIUQ&list=PLWsnao9n727NxWzs8QHsxbhioLxyQOuhb&index=29
https://www.youtube.com/watch?v=FSLkh51bAIo&list=PLWsnao9n727P4FVN44f8LuB9jogjDd0k4&index=31
https://www.youtube.com/watch?v=HUEnjdc3vgU&list=PLWsnao9n727MDQWUUGqHQWPbRKlFmDBq7&index=190

Notes for browsing the channel: The "HD Ultra" sets are the most accurate, but Jet Force Gemini is an exception (it was kind of a beta run).
Some of the HD Ultra sets are 100% sample-perfect (due to the nature of their reverbs): Super Smash Bros., Super Robot Spirits, and Penny Racers.
The rest (except for Jet Force Gemini) are 100% sample-perfect except that their true reverbs drift away from 0 and also cause buzzes and artifacts that I cleaned out for the HD-ification.

The playlist descriptions have links to the raw downloads, which are 48 KHz/32-bit WAV files with a Songs.txt accompanying file detailing each track.

I'll get an update to GoldenEye 007 out soon, but for now enjoy!

L. Spiro

L. Spiro · « **Reply #3 on:** March 14, 2022, 03:45:49 am »

Okay!
There was a bug impacting all of the mono-reverb sets, and I improved my reverb-filtering technology and reverb-harvesting process, and I also changed my export-to-YouTube settings so as not to compress the tracks.
If you heard some imbalances in the GoldenEye 007 set I had released earlier, that is all fixed.
It took 6 long years, but we finally got there!

The Files
https://www.dropbox.com/s/62x3c33yl2oqdg7/GoldenEye%20007%20HDU.rar?dl=0
48 KHz/32-bit.

The Playlist
https://www.youtube.com/playlist?list=PLWsnao9n727PLCC-QTWwhYhEdqXd5GPce

The Samples
https://www.youtube.com/watch?v=amDKp1T3P6U&list=PLWsnao9n727PLCC-QTWwhYhEdqXd5GPce&index=6
https://www.youtube.com/watch?v=uK4kHb22sZE&list=PLWsnao9n727PLCC-QTWwhYhEdqXd5GPce&index=51
https://www.youtube.com/watch?v=p6zbCJoF8LU&list=PLWsnao9n727PLCC-QTWwhYhEdqXd5GPce&index=14
https://www.youtube.com/watch?v=YVUO90EbBTM&list=PLWsnao9n727PLCC-QTWwhYhEdqXd5GPce&index=54
https://www.youtube.com/watch?v=jnPeXcU_2No&list=PLWsnao9n727PLCC-QTWwhYhEdqXd5GPce&index=9

The Inside
I think people find the behind-the-scenes interesting, so here are some peeks!
This is Nintendo Synthy-4, and while I intend to make it public at some point, for now I am organizing the files to convert and using macros to change certain run-time settings between games.

GoldenEye 007 is the only game using NS4_MASTER_REVERB_OFFSET, since all other games bake the tap offsets into their tables.
GoldenEye 007 is an NS4_CURVE=40.0 game while Perfect Dark, Banjo-Kazooie, Banjo-Tooie, Jet Force Gemini, Conker’s Bad Fur Day, and Donkey Kong 64 are NS4_CURVE=20.0 games.
What’s this?
The MIDI standard recommends that cc7 (main volume) be handled using log10(X)*40.0 to get dB, rather than using the standard linear->dB equation of log10(X)*20.0.
The SDK shipped compliant with the standard and later Rare (Graham Smith specifically) gutted the SDK and made this among other changes (at the request of Graeme Norgate).

“Taps” refers to how the reverb routines transport individual samples from the “wet” buffer through a pipeline.
If you sent a single 1.0 dry sample through the reverb process, the resulting output is a map of where the wet buffer was copied and pasted by the reverb routine—the taps that characterize the reverb routine are exposed.

In this image, the top image is Conker’s Bad Fur Day reverb captured using a single sample as above, and the bottom is a filter I apply to re-center the waves.

The individual taps are then harvested into a table.

That large downward tap at the start of the filter image is here in the tap buffer at index 0, offset 192 samples (in the original game’s Hz), value -0.06183050019837032.

Sometimes I regret the choices I have made in my life when my screen looks like this at 3:00 AM!

Some of the internals.

Notice that handling pan doesn’t just change a track pan value, it instead checks for bActive and updates each note instead. This reveals a quirk about how MIDI is handled by the Nintendo 6 base SDK: Panning, volume, and pitch-bend changes are not applied to notes that are in the release phase! Additionally, each active note maintains its own pan, because notes start on whatever the active pan is for a MIDI track, but interpolate when panning is updated. So an active note might be spending 16 samples to interpolate its pan from the old value to the new track pan, but a new note that is triggered will just start on the current track pan. That interpolation is note-specific!

Below that you will see how program changes are handled. Have you ever wondered why you can play back the SubDrag exported MIDI files and the pans and volumes are sometimes—but not always—wonky?
This is the secret sauce: (ui32TickOfLastVolChange != teEvent.ui32Time || bIsTick0)

If you implement these 3 rules strictly, you will be able to get beyond those annoying inaccuracies:
#1: Changing instruments updates volume, pan, and pitch-bend. Pitch-bend is set to 0, while volume and pan are taken from the instrument’s volume and pan properties. These are almost always set to 0x7F and 0x40. These values simply overwrite the track’s current volume and pan properties.
#2: When gathering MIDI events to process during the current frame, the game’s MIDI engine will first look for a program change and handle it first before handling the other events queued up for that MIDI tick. So if a pan and program-change are on the same tick, and the pan is before the program-change in the MIDI file, this means that first the program change will update the pan, volume, and pitch-bend and then then pan event will be handled, which will overwrite the pan set by the program change. Effectively, the program change is sorted to come before all of the other events in the same MIDI tick, but only 1 program change per tick gets sorted up to the front of that tick.
#3: Tick 0 has an implicit program-change to instrument 0.

This means that the default volume and pan for each MIDI track is whatever the instrument pan and volume are for instrument 0. They do not default to 0x7F and 0x40.
And if you combine rules 2 and 3, you get the reason that all Nintendo 64 composers were confused and vowed revenge on whoever programmed the SDK.
As an example, Dan Hess told me that sometimes he couldn’t figure out why he needed to set the pan or volume twice for it to work, and if you browse all of the Rare MIDI files from all of their games you will see that they always leave space at the start of the MIDI file specifically to avoid the dreaded tick 0.

If you are developing the Nintendo 64 SDK MIDI routines, you will realize that when a composer puts a program-change, a pan, and a volume all on the same tick, the intent is to play the given sample at the volume and pan the composer specified on that tick.
But you know that the MIDI data could be coming from any utility, and that those events might be sorted in their MIDI tool to put the program-change after the other events on the tick.
So you sort the events on a given tick to always handle the program change first, regardless of the order of the events in the MIDI file.
And there was much rejoicing. The intent is always expressed correctly.
You know that handling program changes is expensive, so you limit the system to only sorting up to the first program change you encounter—multiple program changes should never be on the same tick.
But now you add that implicit program-change to instrument 0 at tick 0 and now every composer is angry and confused.
The default set-program command causes your routine to not perform the above sorting on tick 0. It stops searching for program-change events after it finds the first one among the queued events, so now tick 0 behaves weirdly when the composer’s MIDI tool puts the tick-0 program change (which was standard for these composers to do before Nintendo 64 chungled their lives) after the tick-0 pan and tick-0 volume, the engine will handle the pan, then the volume, then the program change, which overwrites the volume and pan.

This tick-0 quirk has caused the bugging of many Nintendo 64 OST’s.
For example, Chameleon Twist isn’t a mono OST, it just sets its pan and volume on tick 0 and then those were overwritten when the tick-0 non-sorted program change gets handled.
https://www.youtube.com/watch?v=TX5R8JZ6_B8&lc=Ugx_DDUtJu7_eKkOVgJ4AaABAg See first comment.
In Chameleon Twist 2, Koichi just gave up on figuring this all out and, realizing that the volumes he was getting were tied to the instrument volumes, he just set the instrument volumes to the levels he wanted and composed around that.

There were literally 0 studios using the original Nintendo 64 SDK who knew what the hell what happening and how to handle it correctly (a tool to move an tick-0 program changes to come before any pan/volume events on a given track), so all studios either got bugged by it or worked around it. At Rare the in-house solution was to simply avoid tick 0. This is why you see them setting pans and volume and programs on ticks 1 and later.

However, mistakes could still be made.
Caverns X in GoldenEye 007 has strings that are incorrectly too loud due to all of these quirks.
I may release a fix for this after I have a talk with Graeme about it.

Well I know that most of you are only here for the goods, but I hope that bit of technical exploration was also interesting!
Enjoy!

L. Spiro

L. Spiro · « **Reply #4 on:** June 19, 2022, 06:41:19 pm »

Wake[of]theBunT · « **Reply #5 on:** August 07, 2022, 02:02:33 am »

Really great work. I enjoy these. It's cool that this classic music is still being brought to life in new high quality ways even to 25 years later!

The GE:S versions are amazing. And nostalgic at this point for me, also. Really the answer is there isn't enough times you can make/remake/remaster music for this game

GoldenEye: Source Forums

News:

Author Topic: GoldenEye 007/Perfect Dark HD Music (48kHz/32-bit) (Read 33499 times)

L. Spiro

GoldenEye 007/Perfect Dark HD Music (48kHz/32-bit)

L. Spiro

Re: Nintendo Synthy-4 and GoldenEye 007 HD Ultra

L. Spiro

Re: Nintendo Synthy-4 and GoldenEye 007 HD Ultra

L. Spiro

Re: Nintendo Synthy-4 and GoldenEye 007 HD Ultra

L. Spiro

Re: Nintendo Synthy-4 and GoldenEye 007 HD Ultra

Wake[of]theBunT

Re: GoldenEye 007/Perfect Dark HD Music (48kHz/32-bit)