Some notes on NeurIPS 2024

• Our poster at NeurIPS 2024 🎯
• How to make the most of attending a conference
  • The Goal
  • The Tools
  • The Activities
  • Other Activities

Our poster at NeurIPS 2024 🎯

I’m proud to share that our poster received quite a bit of attention at NeurIPS 2024! Several people intentionally came to our poster to discuss our work, and many others stopped by to ask questions (thanks to the big photo of Taylor Swift 🎤).

I’m also excited to see how our idea of the concept graph 🕸️—introduced in our NeurIPS paper and complemented by our under-review paper—can be connected to many other works, not only in machine unlearning but also in topics like personalization, controllable generation/editing, and model debiasing within Generative AI.

In a nutshell, the concept graph represents a model’s knowledge as a graph, where nodes are concepts, and edges represent the relationships between those concepts. Understanding this graph structure is essential for many tasks, such as:

• Machine Unlearning 🗑️: The goal here is to remove the model’s knowledge of certain concepts while retaining its knowledge of others. The concept graph structure helps identify which concepts are critical to the model’s performance and should be preserved.
• Personalization 👤: The goal is to personalize the model’s knowledge for a specific user. For instance, changing “a photo of a cat before Vancouver Convention Center” to “a photo of a cat before the user’s house.” Traditional methods like Dreambooth, which fine-tune the model on a small user-specific dataset, often overfit to the specific concept and degrade the model’s general capability. Prior approaches address this by collecting large datasets of heuristically selected concepts—e.g., if the personalized concept is “a user’s house,” the preservation dataset would include a variety of house images. Our concept graph structure can help identify which concepts are specific to the user and should be preserved, improving the balance between personalization and generalization.

I also had the opportunity to discuss related works with their authors and promote our concept graph structure. While these works are already impressive, I believe incorporating our concept graph could help improve their results even further. ✨

NeurIPS 2024 marked my first in-person conference, and I must admit it was a bit overwhelming. There were so many excellent works to read more, many new ideas to explore, and a few disappointing moments when seeing some of my future ideas had already been done. 😅 Nonetheless, it was an eye-opening experience, and I guess my upcoming holiday assignment is to wrap up and review the many fascinating works in machine unlearning and personalization. Stay tuned for that! 📚

How to make the most of attending a conference

NeurIPS 2024 marks the first time I attended a conference in person (yes, after joining academia in 2019 as a PhD student). Since it was my first time, besides the excitement of meeting some of the most brilliant researchers in the field, I also felt a bit nervous and overwhelmed. Many things were new to me, so I wanted to note them down here to make the most of any future conferences.

The Goal

NeurIPS is one of the largest and most prestigious conferences in the field, with around 4,500 accepted papers, a packed schedule, and numerous activities over six days (from Tue, Dec 10 to Sun, Dec 15). It’s impossible to attend all the sessions, and the feeling of being overwhelmed and fearing missing out is very real (you can see many people walking around quickly with phones in hand, scrolling through the Whova app).

For me, the most important thing was to set a clear goal. Specifically:

• Goal: Connect with people, make friends, find job opportunities, or learn about the latest research in a specific topic. While some people are very good at achieving all these goals simultaneously, I focused on learning the latest research only at this conference. Somehow, I was not ready for networking activities.
• Topics: Narrow down the topics of interest, such as machine unlearning, personalization, etc.

The Tools

Even with a narrow goal and specific topics, it’s easy to get lost in the conference. Here are some tools I found helpful:

• Whova: The conference app, which is incredibly useful for checking schedules, locating session venues, and managing poster sessions.
  • My Agenda: Replaces Google Calendar during the event.
  • Search in Agenda: Makes it easy to find sessions/posters/activities using keywords.
  • Attendee List: Allows searching for people with specific interests, e.g., Vietnamese researchers by searching for Nguyen in the name. :D
  • My Profile: It’s important to update your profile to make it more visible to others (e.g., Interests, Education, Location). Thanks to this, I was contacted by someone from Databricks who had also been at Monash University and will visit Melbourne next year. We’re planning to meet for coffee!
• LinkedIn and X (formerly Twitter): Use conference tags like #neurips2024 to find interesting events or people. I discovered several job postings this way and also came across this helpful website: https://jalammar.github.io/assets/neurips_2024.html, which simplifies navigating through the papers. It’s also a great distraction when bored. :D
• Google Maps: Download offline maps because you may not always have an internet connection (poor researchers can’t afford roaming fees or visitor 4G SIMs).

The Activities

• Poster Sessions: The most exciting part of the conference, offering a glimpse into the research community as people walk around discussing ideas. While nowaday, we can find any posters or papers we want on the conference site, the energy and excitement of walking around and discussing ideas with authors directly is still unmatched. Moreover, when discussing or seeing the authors, you can also have a feel of their personality and research style that you never get from reading papers. These signals are also important to filter out the good papers from the bad ones. The design of a poster might also give you a hint of the quality of the paper.
• Tutorials and Workshops: These sessions are focused, sometimes small (not true for NeurIPS), and directly related to your topics of interest.
• Oral Sessions: Featuring the best papers of the conference, these sessions are inspiring. In NeurIPS 2024, I found three oral presentations particularly fascinating. What inspired me most was that all these works were based on simple yet effective ideas—proving that you don’t have to be a hardcore mathematician or statistician to produce great work at NeurIPS.
• Job Fair - Sponsor Sessions: An excellent opportunity to find job openings, learn about industry research/products, and grab freebies! This year, I saw many demos on LLMs, such as Amazon’s debugging assistant on AWS and Microsoft’s trace agent tool. A startup called Virtue, founded by prominent researchers in the Trustworthy AI community, showcased their automated red-teaming solution.

While many people enjoy networking activities like Coffee Chats or Company Parties, I wasn’t ready for those and felt a bit shy about starting a conversation.

Other Activities

• Visiting Friends: I was fortunate to catch up with two old friends from Monash University and Viettel in Vancouver, chatting about our old days.
• Exploring the City: Vancouver is a beautiful city with a stunning harbor view. I also have a chance to see the dark side of the city with many homeless people and drug addicts.
• Staying Healthy: Jetlag hit me hard after a 19-hour flight from Melbourne. I didn’t do well in maintaining my health this time (poor sleep, too much curry and coffee, and lack of exercise). I’ll aim to do better next time.