2022-10-30: Update 11/75 - NVIDIA DGX-2 and Swiss Cheese

Such a cheesy title! The reason for the title is: The thermal discharge of our chair’s DGX-2 (explanation below) helps heat a Swiss cheese factory (no joke).

I can thus proudly proclaim to be a contributor to Swiss cheesemaking. This week, parts of my code ran on our DGX-2 for the first time.

We had a spontaneous idea for a paper and ran experiments for this idea on the DGX-2. As mentioned in earlier updates, we are planning to create a dataset. The experiments we ran last week examine how existing AI benchmarks perform with less training data. With these experiments, we want to determine which minimum training size we need when creating our own dataset.

Here is the explanation of what a DGX-2 is: A DGX-2 is a hardware system by graphics processing unit (GPU) manufacturer NVIDIA. It contains 16 NVIDIA V100 GPUs, which are individually already strong GPUs. DGX-2 is specifically designed for artificial intelligence research.

You can follow these updates: Substack Blog Telegram WhatsApp LinkedIn Medium Twitter Calendly

^{Say cheese!}

What Happened Since Last Week?

We ran scaling experiments to see how question answering systems perform with different training data sizes. We have set up our experimental structure and the code, and now we need to expand the experiments to additional datasets and models.

What Were the Biggest Obstacles?

We changed the path in an unforeseen direction. I started the week without a clear goal in mind, and is now running experiments on dataset scaling was a shot in the dark. We will see if this turns out to be a good path.

Which Goals Did I Meet?

No goals were set, so no goals can be met.

Which Goals Did I Miss?

I leave this goal here as we are still not 100% sure:

Decide on whether to prepare a dataset ourselves or take an existing dataset.

Was It a Good Week?

Yes, at least I was productive. The chosen path was spontaneous, and the work might have been wasted. Let’s see what our experiments will yield.

Short-Term Tasks for The Coming Week

Expanding the experiments to additional models.
Optional: Expanding the experiments to additional question answering datasets.

About “75-Step Journey Toward a Ph.D. in Natural Language Processing”

You will, from now on, witness my grind. Feel my blood, sweat, and tears.

With this series of articles, you become a real-life weekly witness of my dissertation progress, all in 75 steps. This has multiple purposes:

1) Forcing myself to keep moving through the power of public shame!

2) Helping other (prospective) Ph.D. students to stay motivated and to show that hard times are normal when going through this process.

3) Getting support from the community when I go through hard times.

Share this with your Ph.D. student friends: Substack Blog Telegram WhatsApp LinkedIn Medium Twitter Calendly.