Skip to main content

Posts

Showing posts from August, 2015

What Ifs? Sigmoid Function vs. Error Function in Machine Learning through Logistic Regression

While brushing up on some study materials in Mathematics, a familiar function piqued my interest. It was the error function, the solution to the non-elementary integral exp(-x^2) and whose complement is used in determining the conditional probability of bit error due to noise:






Or quite simply, the probability of error due to noise.




But the real point of interest was the nature of the curve of the function shown below.



Now, why be so interested in such a function? When I compared erf(x) with the sigmoid function commonly used in defining the decision boundary in machine learning algorithms, it returned a steeper slope. Then the thought came to me. What would be the differences of using the error function instead of the sigmoid function? Would the cost improve? Would the training accuracy improve?

And so my curiosity got the better of me and I played around with both of the functions to see what would happen.

Sigmoid vs. Error
First of all, replacing the sigmoid function with the error…

Sonata Pathetique [Allegro di molto e con brio - Part 2]

Took quite a while to upload part 2 of Sonata Pathetique given the schedule of an engineer in the industry (still an amateur video)... The entire piece would've cost me a 1 Gb upload! Current focus of practice is only on Hanon, as typical classical and concert pieces gravitate on the exercises it contains. Hope it doesn't sound to bad. Hehehehe... ^_^

Inconspicuous Pitfalls in Datasheet Analysis

A chess grandmaster, who also happens to be an electronics hobbyist, decides to design his own chess clock. The LEDs, normally-open push-button switches, and microcontrollers are all in place, driven by a pair of 1.5V AA batteries. But the clocks only last for a few hours. So he checks online for a power management IC to extend the battery life, and finds one promising an efficiency of 94%. After purchasing the chip and ramming it onto a prototype, he becomes puzzled when the operating lifespan doesn’t change much. He then checks the wires and solders and connections again in a vain effort to find a solution.

Let us take this to the industry level. Apple is designing its 23rd Iphone. The company producing the processor for the Iphone23 is looking for a PMIC that can deliver a transient response of around 10mV with a 10A shifting load (Absurd!). They contract a semiconductor company to design the chip. After numerous tedious design reviews, the PMIC becomes ready. When the chips are t…