1. Fundamentals of Data Engineering This book dives into the world of data engineering, covering everything from how data is collected and stored to how it’s processed and used. It talks about best practices, security, and designing systems that handle data efficiently and safely. 2. Four Thousand Weeks This book Read more…
Link to paper https://arxiv.org/abs/2308.14296 Notes The paper propose a novel algorithm, Self-Inspiring, to improve the planning ability of the LLM agent. At each intermediate planning step, the LLM “self-inspires” to consider all previously explored states to plan for next step. Literature survey Architecture Tools they used 1) DB tool1) To Read more…
Introduction to Building Machine Learning Applications Building machine learning applications requires a thorough understanding of the fundamentals of machine learning and software development. This section will provide an overview of the key considerations and best practices for building machine learning applications. Best Practices for Data Preprocessing Data preprocessing is a Read more…
When I was a junior engineer I was given small bugs to fix and small features to work on. However the way we develop larger features is different from the way smaller bugs are fixed or features are developed. I have worked at couple of big names at the current Read more…
It is very important to identify everyone’s priorities in a software engineering organization. In a sense, priorities are about people and what facts are about things. Because of this, it can be easy to overlook priorities, since they are more difficult to ascertain than simply reading documentation or playing around Read more…
A C++ string class can be implemented in more than one way. I used char*(or rather T*) as a base for implementation. The reason for templatization was to mimic the way standard library does things. It supports all the construction, copy and assignment operations as well as outputting to stdout Read more…
For binary classification problems, a good way to measure the performance of a model is by finding out AUC (Area Under The Curve) of ROC (Receiver Operating Characteristics). What is a ROC curve? It is a plot of True Positive Rate(TPR) vs FPR at various thresholding levels. Let’s understand this Read more…
Confusion Matrix Confusion Matrix is a group or matrix of metrics in supervised learning scenarios which determine how good a model is in predictions. Lets consider a binary classifier with results 1 or 0. A confusion matrix here will be a 2×2 matrix where along one axis you have actual Read more…
Hypothesis Testing Hypothesis tests, also called significance tests, are ubiquitous in the traditional statistical analysis of published research. Their purpose is to help you learn whether random chance might be responsible for an observed effect. KEY TERMS Null hypothesis The hypothesis that chance is to blame. Alternative hypothesis The counterpoint Read more…
This is part 2 of Introduction to Dimensionality Reduction. In this blog post, we would several different mathematical prerequisites that one must know before trying to understand machine learning. Mean Vector The sample mean is a vector each of whose elements is the sample mean of one of the random Read more…