Tin Sum Cheng

Tin Sum Cheng

Dedicated PhD student in machine learning theory with a robust mathematical foundation. My goal is to contribute to cutting-edge research, leveraging my mathematical expertise and my interest inmachine learning to drive innovation.

Blog Post: Neurips2023 Day 2

1 minute read

Published:

Conference blog.

I am attending Thirty-seventh Conference on Neural Information Processing Systems (Neurips2023) now and I am happy to share my experience with you all. I will write a short summary every day.

Learn like a baby

Babies are curious fast learners! I wish that I behave always like a baby in terms of learning.

Talks I have attended

TLDR: Babies prefer looking at simple patterns and big contrasts in the first few months upon their birth. But they are great one-shot learners: for instance, they know the word “banana” even after they have seen it only once and before they can speak. This incredible learning performance may come from their biased learning behaviour so that they can capture features very fast.

Comment: This is a great reminder for machiner learning scientists. Maybe we should no longer assume i.i.d. or Gaussianity of the data input as the data in this world is structured and it is at best to learn them with bias! Great talk from the view of biology/psychology which brings great insight in machine learning.

TLDR: The award is given to the original paper of words tokenization in 2013. 10 years has past and this serves as the foundation of the success of LLM.

Comment: This is a good example of how brilliant ideas can be built upon each other and the success of machine learning is not a coincidence but a long process of experiments.

TLDR: Invited speakers talk about the perspective as christians, Jewish, Muslim and indigenous towards the development of AI, especially on privacy, generative models, consciousness and language models.

Comment: It is an interesting perspective for AI ethics.

Extras

  • I attended a poster session today and met many researchers, known or unknown before, and learn a bunch of new stuff!
  • I had lunch with new friends from different fields and sectors, I am so inspired by their thoughts and works!
  • Is it normal to enjoy the conference more than a vacation?