Fundamentals of logistic regression and concepts like maximum likelihood function, cross-entropy are covered briefly

photo taken from knowledgehut

Logistic regression is one of the most commonly used algorithms for machine learning and is a building block for neural networks. It is important to understand the fundamentals and maths behind logistic regression. In this post, we will explore the fundamentals of logistic regression, and concepts like maximum likelihood function, cross-entropy/log-likelihood are covered briefly. In the end, we have provided codes for Logistic Regression in Python from scratch.

Previously, we have covered linear classifier models and we have described the concept of decision boundaries. In linear classifier models, we take some features xᵢ, some random weights wᵢ and calculate the…


Descriptive and summary statistics associated with histogram, boxplot, and scatter plot are covered briefly.

Statistics is very important in our day to day life. There are so many applications of statistics in sectors like predictive analytics, machine learning, etc. It is really important to understand statistics well in order to understand these high-level concepts. In this post, we will introduce different parts of statistics, and then in later posts, we will describe them briefly.

Statistics

Statistics is the discipline that concerns data collection, organization, analysis, interpretation, and presentation. So what is data? Data is simply defined as distinct pieces of information that can be in many forms e.g. text, video, databases, spreadsheet, audio, images, etc…


Photo by Will Myers on Unsplash

Mathematical and statistical background of Various outlier detection methods with a detailed comparison between the use cases

Most real-world data sets have outliers that have unusually small or big values than the typical values of the data set. Outliers can have a negative impact on data analysis if not detected correctly and also outliers can reveal significant information and characteristics of data. There are several methods to detect an outlier in the data set. It is important to understand the characteristics of these methods properly since they might be quite powerful with large normal data but it is problematic to apply them to nonnormal data or small sample sizes without proper knowledge of their characteristics. …


Why regularization is used in machine learning and how it reduces overfitting balancing bias-variance

Image by OpenClipart-Vectors from Pixabay

In the earlier era of machine learning, there used to be a lot of discussion on the bias-variance trade-off as was we could increase bias and reduce variance, or reduce bias and increase variance. But back in the pre-deep learning era, we didn’t have many tools that just reduce bias or just reduce variance without hurting the other one.

But in the modern deep learning, neural network or big data era getting a bigger network almost always just reduces bias without necessarily hurting variance, so long as we regularize appropriately. Also, getting more data always reduces variance and doesn’t hurt…


A detailed explanation of Mathematics and concepts behind Neural Network

Neural Network has become a crucial part of modern technology. It has influenced our daily life in a way that we have never imagined. From e-commerce and solving classification problems to autonomous driving, it has touched everything. In this article, we are gonna discuss all the important aspects of neural networks in the simplest way possible and at the end of the tutorial, we have also provided code implemented in python for all the described parts.

Motivation

Animal brains even the small ones like the brain of a pigeon was more capable of than digital computers with huge processing power and…


Linear Classifiers are one of the most commonly used classifiers and Logistic Regression is one of the most commonly used linear classifiers. The concepts we are going to learn here will actually extend a lot of other classification methods beyond linear classifiers. We’re going to learn the fundamental concepts, but also the underlying algorithms that let us optimize the parameters of this model to fit your training data.

Motivation


Previously, we have discussed briefly the simple linear regression. Here we will discuss multiple regression or multivariable regression and how to get the solution of the multivariable regression. At the end of the post, we will provide the python code from scratch for multivariable regression.

Motivation

A single variable linear regression model can learn to predict an output variable y when there is only one input variable, x and there is a linear relationship between y and x, that is, y≈w₀+w₁*x. Well, that might not be a great predictive model for most cases. For example, let’s assume we are going to…


Regression is a very fundamental concept in statistics, machine learning, and Neural networks. Imagine plotting the correlation between rainfall frequency and agriculture production in high school. Increase in rainfall generally increases agriculture production. Fitting a line to those points enables us to predict the production rate under different rain conditions. It was actually the very simplest form of linear regression.

In simple words, regression is a study of how to best fit a curve to summarize a collection of data. It’s one of the most powerful and well-studied types of supervised learning algorithms. In regression, we try to understand the…


To build an accurate machine learning model we need to have a proper understanding of the error. In forming predictions of a model there are three sources of error: noise, bias, and variance. Having proper knowledge of error and bias-variance would help us building accurate models and avoiding mistakes of overfitting and underfitting.

In this tutorial, our case study will be how to predict house prices. We have a dataset which consists of house prices with the square feet of the house associated with it.

Noise

Imdadul Haque Milon

A mathematician became AI enthusiast and blogger!

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store