https://NLTuan.github.io/2025-09-24T00:00:00-04:002025-09-24T00:00:00-04:002025-09-24T00:00:00-04:00Le Tuan Huy (Tony) Nguyentag:nltuan.github.io,2025-09-24:/what-i-am-working-on-currently.html

<p>It has been a few months since my first and only blog post, and I'd love to get back into writing and sharing once again. I know that writing does me a lot of good in verbalizing my thoughts clearly. Since it has been so long, this blog is an update to what I have been working on throughout this summer.</p>

<p>It has been a few months since my first and only blog post, and I'd love to get back into writing and sharing once again. I know that writing does me a lot of good in verbalizing my thoughts clearly. Since it has been so long, this blog is an update to what I have been working on throughout this summer.</p> <h2>Vision Language Action Models (VLAs) and Robotics</h2> <p>I am grateful to have been given the opportunity to work in an ML robotics project, which was something that I haven't anticipated beforehand. This project consists of controlling a robot to perform an action based on a verbal input. This leverages VLAs that produce actions that predict what actions the robot should take. VLAs are composed of 2 parts, a VLM (Vision Language Model) that processes text and image inputs and a Diffusion model that generates multiple actions. The architecture that we are currently using is <a href="https://huggingface.co/blog/smolvla">SmolVLA</a>. The <a href="https://github.com/huggingface/lerobot">lerobot</a> library is great for most of the tasks that we need to do. They include: recording data, training a policy, and running asynchronous inference. The robotics space is riddled with non-AI problems with hardware and Linux that I have had the privilege to never encounter beforehand.</p> <h2>LLM finetuning research</h2> <p>I am also active in LLM finetuning research with the Local Research Group in the <a href="https://course.fast.ai/">fast.ai</a> Discord. This project consists of comparing the efficacy of different finetuning techniques (full finetuning, LoRA, rsLoRA, DoRA) on domains of math and coding. We are looking for improvements whilst also retaining base model capabilities. I was tasked with model evaluation. For this, I used <a href="https://github.com/EleutherAI/lm-evaluation-harness">lm-evaluation-harness</a> for evaluation with VLLM support. Other ongoing tasks within the team are: custom modeling for efficient training, data decontamination, chat templates, etc. </p> <h2>Personal Learning</h2> <p>On the side, I am learning RL algorithms alongside Clusters of Stars, a group within the fast.ai Discord group. I have learned and implemented DQN (Deep Q Networks), Policy Gradient, and A2C (Actor-Critic) from scratch with PyTorch: <a href="https://github.com/NLTuan/rl_imps">RL Implementations</a>. This was to build my RL foundation that I have largely disregarded due to the scary math. Given enough time thinking and coding, these scary math notations become slightly less scary as my intuition develops (still super scary though!). This RL knowledge is crucial towards understanding how modern RL for post-training (PPO, DPO, GRPO) works.</p> <h2>Looking Forward</h2> <p>These experiences have been incredibly valuable in expanding my technical skills while also teaching me the importance of persistence when facing complex mathematical concepts. The intersection of robotics, LLMs, and RL continues to fascinate me, and I'm excited to share more detailed technical posts about these projects in the future.</p>2025-04-18T00:00:00-04:002025-04-18T00:00:00-04:00Le Tuan Huy Nguyentag:nltuan.github.io,2025-04-18:/welcome-to-my-blog.html

<p>Hello to whoever is reading this! You're currently on my first ever blog post! I am no big writer and this is my first genuine attempt at creating and publishing posts that are meaningful to me. Writing has always been intimidating to me, but this will serve as a starting point towards being more comfortable putting my ideas into words and sharing them online. The main reason why I have decided to start blogging seriously is that I want to value creation over passive consumption and to keep track my learning and ideas.</p>

<h1>Welcome to My Blog</h1> <p>Hello to whoever is reading this! You're currently on my first ever blog post!</p> <p>I am no big writer and this is my first genuine attempt at creating and publishing posts that are meaningful to me. Writing has always been intimidating to me. I've always hated, it but this will serve as a starting point towards being more comfortable putting my ideas into words and sharing them online to super duper cool readers like you.</p> <p>The main reason why I have decided to start blogging seriously is that I want to value creation over passive consumption and to keep track my learning and ideas.</p> <h2>Who am I?</h2> <p>Currently, I am CS student entering McGill University in Montreal Canada. I'm interested in AI, specifically anything related to LLMs whether that be research or engineering. I love to train models! I try my best, but it is a daunting task with many pitfalls. I am also a RAG enthusiast; I'd love to learn more about it and develop apps or systems.</p> <h2>What to expect</h2> <p>There is nothing set in stone for the near future as of writing this. Although one promise that I'll try to uphold is sharing technical things like my learning journey in AI and software development or my random thoughts on random things.</p> <hr> <p>You've reached the end of my first ever blog post. This was just raw words from me. One day I will come back to this and cringe at it probably, but it is the risk that I am willing to take if the reward is becoming a better communicator.</p> <p>Thanks for reading :)</p>