The Learning Dynamics of a PhD

23 Dec 2024

For a change, let's turn our attention from the learning dynamics of Language Models to a more human-centric perspective—specifically, by tracking the time spent on my own PhD down to the second.

This is what a PhD looks like: 1347 days total, of which 111 days 2 hours 3 minutes and 59 seconds were spent exclusively on research that went into the final dissertation. Let's take a closer look at how we got here, and how this could help you (and especially new PhDs) in your own long-term endeavors.

Methodology

First, we need to establish what is being measured (quantifying things is kind of my thing; see Müller-Eberstein, 2024). What the heatmap above aims to reflect is the total research time invested into the PhD project.

The overall duration of 1347 days (=number of squares) stems from the total number of days I was employed under my PhD contract, i.e., from my first day of work (💐) to the day I handed in my dissertation (📕). Note that this does not include the non-negligible amounts of time and sweat spent on applying to positions, as well as preparing and conducting the PhD defense itself.

Additionally, we only track PhD research time, so no courses, teaching, administrative tasks, etc. Similarly, while disseminating research and networking at conferences are both important parts of a PhD, these are also excluded. Finally, to delimit the target a bit further, we also do not count time spent on collaborative projects, which—while extremely fulfilling—are not directly included in the final thesis.

With these annotation guidelines in mind, we now hit start on the time tracker. Specifically, I used the built-in Focus function of my beloved GoodTask task manager, which keeps track of start/end times per project, and exports these data into neat little JSON files. Of course, there are time tracking apps with more granular features, but for my purposes this is sufficient. Thanks for keeping me sane, GoodTask! 🧡

Hyperparameters

Pre-training. Preceding the PhD, we have parental guidance, general education and a later specialization in Computational Linguistics (BA), plus further fine-tuning in Machine Learning and Artificial Intelligence (MSc).

Training. In Denmark, one epoch of PhD training covers three years of full employment exactly. The batch size is 37.5 hours per week, with 7.5 hour micro-batches including lunch. The learning rate schedule is all over the place.

Infrastructure. My ELLIS PhD was conducted on distributed hardware across the IT University of Copenhagen's NLPnorth group, LMU Munich's MaiNLP lab, and Ivan Titov's lab at the University of Edinburgh. You can check the CO2-eq emissions of each respective country on Electricity Maps, with additional offsets from a 15-year vegetarian diet.

Role Models. I was extremely fortunate to have Barbara Plank, Rob van der Goot, and Ivan Titov as supervisors. Not only were they excellent role models to learn from, but they are also awesome humans in general. I hope you can also find advisors who you can learn from and vibe with—something that, in my opinion, is critical to a successful PhD.

Limitations

Before continuing, a few words regarding the limitations of this analysis: First, 61 days are missing time tracking data due to (ehem) experimenter oversight. For these days, I sample estimated hours from a truncated normal distribution conditioned on the weekday in question, centered around the mean hours and standard deviations for each weekday with measurements.

Second, this analysis applies to one very specific PhD in a specific field in a specific place. Overall, I would say that the PhD duration of three years in Denmark is quite short, and I have no clue how to complete a dissertation in a field where it is not possible to iterate as quickly as in Computer Science. What I can say though is that pausing your contract (e.g., with internships) definitely helps break up the total time. Even if you're working on something other than your PhD during those times, it spaces out any paper submission deadlines.

Third, there are countless confounding factors beyond those listed in the hyperparameters. There may be some advice that applies to PhDs in general, but every situation is different. Not least of all, life keeps happening, and should always take precedence over your PhD work. Take care, and go on walks.

Analysis of PhD Learning Dynamics

Mirroring the learning dynamics of language models, let's split up the PhD into initial, critical and specialization learning phases, before we turn to some outliers and higher-level take-aways.

Here, it helps to look at the cumulative time spent on PhD research in relation to how much time has passed in the overall contract. The diagonal through line represents an ideal case in which we would have spent exactly the amount of time on our PhD that is proportional to the time left in our contract (i.e., 50% of work done half-way through the contract). Note that we can only measure this ratio in hindsight, as it's impossible to know beforehand exactly how long the whole endeavor will take.

Additionally, the smoothed average hours per day are plotted above to visualize the intensity of work in each period.

Initial Learning Phase

In Y1, and the first two months in particular, you can see a gradual, but intensive increase in PhD hours. As you can imagine, this time is spent getting set-up both mentally (e.g., memorizing everyone's names) and physically (e.g., perfecting the cable management on your desk). Working out the administrative stuff also takes up a lot of time. At ITU for instance, we need to submit a rough plan of our entire PhD within the first three months of starting.

While this keeps you busy, there are few other hard requirements, so it's also an excellent time to explore, ideate, and hit the ground running. A thing to keep in mind is that my PhD was project-based, so although there is also a lot of freedom to shape the research to your interests, my supervisors already had ideas for general directions.

Critical Learning Phase

The Y2 + the first half of Y3, I would categorize as the critical learning phase. With relatively few external requirements, I was able to fully dive into the broad direction set out in the initial learning phase, and to explore multiple avenues. It's an excellent time to follow conferences closely (even if you're not presenting), and read a lot, to understand the open questions of the field, and to crucially also understand what interests you.

At ITU, we have a sort of pre-defense called the "midway evaluation" (other universities may have similar structures, such as a one-year review etc.). For me, the midway fell into month three of Y3. While it sounds like a admin-heavy exercise, writing up a 10-page report on what you did, how it fits together, and what you plan to do next was super helpful in identifying the core questions of interest, and allowed me to use my remaining time more effectively for specialization.

Specialization Phase

Interestingly, the most productive year in terms of publications is not the one in which I dedicated the most time to research. Y3 was mostly spent disseminating the research coming from the critical learning phase, and using conferences as opportunities to identify where to go next in the area that was of most interest to me.

Looking back at the cumulative time spent, at this point—around 50% through the contract—I had already invested 60% of the time I would be spending overall into PhD research. This buffer allowed me to go into an internship in Y3 with a good conscience. Of course, hindsight is 20/20, and it's impossible to know when you've hit that 60% mark a priori. But I would guesstimate that once you have found a specialization that is valuable to your field, interesting to you, and the elevator pitch rolls of the tongue easily, you've reached that point.

With other obligations, collaborations, etc. piling up, it becomes more difficult to dedicate as much time as in the critical learning phase to your PhD, but knowing where your goal is allows you to double down and work on your specialized interest more effectively. For me, this manifested in a less contiguous, but still very intensive bout of PhD research in Y4—including a productive and fulfilling research stay abroad in Edinburgh (with a lot of cherished coffee breaks), and time for another internship with a different, but nonetheless extremely interesting topic.

Finalization Phase

With these building blocks in place, it's time to finalize the PhD project. You can see the final push towards the dissertation starting—as all good intentions—at the beginning of the new year in Y5. It's important to note that my dissertation is somewhere between a collection, and a monograph, so most of the writing work is dedicated to intros and outros tying the intermediate research together. Here, it once again helps to have a red thread set relatively early on.

For some this thread flows naturally from the progression of their PhD research (e.g., literature review → dataset → method → evaluation), while for others (including myself), it's a bit more difficult, and is more about highlighting different angles to the underlying research questions and themes. Also, while I think that a collection-style dissertation is better suited for a three-year PhD, some colleagues have managed to write a full monograph (with excellent results!), so discuss with your supervisors what works for you.

Finally, while I would have liked to lock myself in a remote cabin, reflecting on my PhD while wistfully staring at the (flat) Danish landscape, there's always other stuff that needs to happen: e.g., applications for after the PhD, conferences, collaborations, and all that. You can see this reflected in short <1 week phases, which are typically followed by frantic catch-up.

All in all, the writing process was surprisingly chill—with a final hand in (read: undramatic upload button click) in lovely Italy during LREC-COLING 2024. Time! ⏱️

Takeaways

Outliers

What's with those very obvious chunks of time where no PhD research happened? First, the largest happenings were summer internships in Y3 and Y4. While you can (and should) aim for internship projects that can be integrated into your PhD, it's also important to remember that internships are the best way to learn something that is outside of your typical academic bubble. Personally, I found it helpful to have a clear separation between university and industry work, and to fully dedicate my time to the new environment and projects at hand. Not only does it help for making new connections, but the overall learnings also transfer to academic research—especially in the field of AI.

Some university work during internships was unavoidable. And although this meant working late into the night due to timezone differences and deadlines, I would recommend going completely offline from university work as much as possible. It makes the whole internship experience much more fulfilling.

Most other times of lower PhD research intensity can be attributed to conferences (although these technically contribute massively to your overall career), as well as vacations towards the end of each year for me (although these shouldn't really count as outliers either).

Work Life Balance

Denmark takes work-life balance very seriously. So let's check out my report card:

Nice! Hardly any work on weekends 🎉 If so, it was centered around a few days before deadlines—especially before a deadline during the Y4 internship. Needless to say, you should never sacrifice weekends for your PhD. Otherwise, there's slightly more research happening on Mondays, and a slight dip on Wednesdays, although I'm hard pressed to pinpoint why (hides coffee mug).

In the overall heat map, one can also see that more time is dedicated to research towards the beginning of each year. This is also reflected in my publication schedule towards end-of-year conferences. With new year's being the most important family occasion in my household, you can also clearly see the holiday wind-down towards the end of each year.

In terms of vacations, I have to deduct points for not taking enough in the first two years. Note that a fully colored-in year should not be what your heat map looks like. The following years, do have vacations towards autumn, but never in the summer—taking the internships into account. Compared to many other countries, Denmark has very forthcoming holiday schemes, so in hindsight, I could have used them more, as I found rest significantly improves creativity down the road.

Now, let's take a look at one final plot:

As stated in the beginning, a PhD is not the only thing happening in this period. Life also keeps happening! While the PhD legitimately is an important part of your life during this time, it takes up only a fraction—specifically, 8.2% in this particular case.

With respect to work, the PhD does rightly take up the majority of the working hours, constituting 57% in total. Put another way, 333.26 working days—or about one year out of the three-year contract—are dedicated solely to PhD research.

This, of course, pales in comparison to the second largest factor: sleep. Here we're assuming an idealized eight-hour rest. And while I am guilty of not hitting this mark as often as I should, it's still worth noting that sleep plays a huge role in your physical and mental well-being as it's the foundation to everything else.

Finally, life—with all its ups and downs—still constitutes the largest portion of time at 44.1%. I found visualizing the time ratios like this quite enlightening, since the PhD does feel like it takes up much more space in your life. Since life here includes both the nice and not so nice things, it's really worth reflecting on how much extra stress one should carry over from work, and how to prioritize. Hopefully, this also highlights how important a supportive environment, and good time management can be to make the impact the PhD has on your life a positive one.

Conclusion

Every PhD is different, but here are some high-level takes from exploring my own data:

Explore a lot during your initial and critical learning phases. Whatever your university administration tells you, use this early phase to focus as many of your work hours as possible on exploring and finding what is of interest to you and your community. (Most admin forms can wait.)

Find your through line as early as possible. There's always more to look into, but try to narrow down the red thread for your current dissertation using the initial exploration phase. Here, an administrative hurdle like a one-year/midway evaluation can actually be a lot of help, because it forces you to bundle your thoughts and place them into a larger narrative.

Be aware of your surroundings. Hopefully, hitting those previous two milestones will give you that "60% done"-feeling, and the freedom to look up from your PhD and check out the surroundings. Be it career-oriented endeavors, such as internships, or the time to appreciate other things and people in your life, try to keep in mind that a PhD is 9.6 million seconds, but only 8% of your overall day-to-day life.