Thankfully in answer to the question of whether a stable matching exists, we have a theorem that guarantees exactly that! It states that:
“For any number entities of two disjoint sets (having equal number of elements), no matter how they rank each other, there always exists at least one stable matching”

While the solution is stable, it is not necessarily optimal from all individuals’ points of view. The traditional form of the algorithm is optimal for the initiator (suitor) of the proposals and the stable, suitor-optimal solution may or may not be optimal for the acceptor of the proposals. It will be acceptor pessimal.

To find the solution, we have the Gale-Shapley Algorithm. In 1962,  David Gale and Lloyd Shapley proved that, for any equal number of suitors and acceptors, it is always possible to solve the Stable Marriage Problem.

The Gale-Shapely Algorithm:
Each suitor proposes to his highest preferred acceptor. If the acceptor is not already matched, we automatically match suitor to acceptor. If the acceptor is previously matched; acceptor picks the most preferred suitor. The rejected suitor moves on to his next preferred acceptor. When each suitor is matched the problem is solved.

Download

Download source code. It is an implementation of the algorithm in .NET 4 using Visual Studio 2010 and C#.

/Here Optimal Entity is the Suitor and Pessimal Entity is the acceptor
//The function that implements the Gale-Shapley algorithm in .NET is shown
public static SortedDictionary<T,U> Match(HashSet<SuitorEntity<T, U>> optimalEntities, HashSet<AcceptorEntity<U, T>> pessimalEntities)
        {
            Dictionary<AcceptorEntity<U, T>, SuitorEntity<T, U>> matching = new Dictionary<AcceptorEntity<U, T>, SuitorEntity<T, U>>();
            Stack<SuitorEntity<T,U>> processStack = new Stack<SuitorEntity<T, U>>(optimalEntities);
            while(processStack.Count>0)
            {

                //Store each suitor in a Stack.
                SuitorEntity<T, U> suitorEntity = processStack.Pop();
                // Match the optimal entity to its first preference
                //If its first preference is not previously matched then automatically match and dequeue
                SuitorEntity<T, U> matchedEntity;
                AcceptorEntity<U, T> acceptorEntity = suitorEntity.Preferences.Dequeue();
                if(!matching.TryGetValue(acceptorEntity,out matchedEntity))
                {
                    matching.Add(acceptorEntity, suitorEntity);
                }
                else//Else
                {
                    foreach (SuitorEntity<T,U> entity in acceptorEntity.Preferences)
                    {
                        if(matchedEntity == entity)
                        {
                            //The Pessimal Entity makes a choice: If it already has a better preference then it keeps it
                            processStack.Push(suitorEntity);
                            break;
                        }
                        // Replace the matching of the acceptor with the most preferred new suitor
                        if (suitorEntity != entity) continue;
                        matching[acceptorEntity] = suitorEntity;
                        //Push the previously matched suitor who is now rejected on to the stack for further processing
                        processStack.Push(matchedEntity);
                        break;
                    }

                }

            }
            SortedDictionary<T,U> result = new SortedDictionary<T, U>();
            foreach (KeyValuePair<AcceptorEntity<U, T>, SuitorEntity<T, U>> keyValuePair in matching)
            {
                result.Add(keyValuePair.Value.Entity,keyValuePair.Key.Entity);
            }
            return result;
        }

References

Interactive Flash Demonstration of Stable Marriage Problem

Stochastic Algorithms are primarily global optimization algorithms. A stochastic process is one whose behavior is non-deterministic. The system’s subsequent state is determined both by the process’ predictable actions and by a random element. The main strategy (with some exceptions) are based on sampling the search space, calculating the ‘cost’ and using that to improve on the result. This can be either through a random process, changing search direction or step size. There is no ‘inspiring’ system, either natural or biological on which these are based.  The first part details the implementation in python of four algorithms: Random Search (RS), Adaptive Random Search (ARS), Stochastic Hill Climbing (SHC) and Scatter Search (SC).  Out of these except for Stochastic Hill Climbing , the remaining three use the same objective function for calculating the cost.

The objective function I chose for RS, ARS and SC was a variation on the one provided by Jason. He used a function that was a ‘sum of squares” , I used a variation that would provide a non trivial solution value, while keeping the structure of the function the same.

Objective function used in Clever Algorithms

Objective Function that I used

The optimal solution (minimum) for  my basin function can be found at the value -10. The input value would be 2.

For SHC I used the same ‘One Max‘ Problem from the original Clever Algorithms book.

Download

Download source code. Unzip and import the attached folder in to your Eclipse workspace. It includes a test suite for exercising the algorithms

Random Search (RS)

Random search has no memory of previous search candidates. So it might revisit previously considered candidate solutions. The advantage is, it is a minimal solution. Its worst case performance is worse than a straight linear approach. But it has its uses in a domain where the values are continuous. Most commonly it is used as an embedded heuristic in other search algorithms for refining a local optima. It is one of the building blocks for other more advanced search techniques.

Adaptive Random Search (ARS)

Adaptive Random Search seeks to avoid the problems of localized random search by sampling with larger or smaller step sizes. We keep trying larger step sizes as long as we have improvement in the result. We go back to smaller step sizes if we don’t see an improvement in the result after a preset max count. The initial starting point is selected similar to that of RS.

Scatter Search (SC)

Scatter search is a global optimization algorithm, which has some Evolutionary computation features. The premise of SC is that the desired result is present in some combination of a diverse set of high quality candidate sets. By combining and recombining the values from a reference set of elite solutions to create ‘offsprings’. We then test these offsprings for best values by applying the objective function to them. We use an embedded heuristic of RS for refining a local candidate solution.

Stochastic Hill Climbing (SHC)

Stochastic Hill Climbing iterates the process by randomly selecting a neighbor of a local candidate solution and evaluating it. It is an improvement on RS, where we don’t sample randomly, but near neighbors of randomly selected candidate solution. I noticed an error in the code listing provided by Jason Brownlee. Specifically on page 40 last line. ( mutant[pos] = (mutant[pos]==’1′) ? ’0′ : ’1′ ) . It should instead check for ’0′ and replace it with ’1′. I believe it is a typo in the book. I also tried to evaluate the search efficacy for the OneMax problem. Since the idea is to replace all zeros in the bitstring with 1. A search efficacy metric can be defined as (FinalOneMaxCount – InitialOneMaxCount)/finalIteration. If we set our max iterations to the length of the initial bitstring, we get a nice metric for measuring the search efficacy.

Visualization of the two objective functions:

Visualizing the original basin function

Visualizing my basin function

My first attempt at reading through it was frustrating to say the least. It was not Jason’s writing style per-se, but the nature of the subject itself. It is not amenable for reading as a programming book. Jason himself cautions in the first paragraph: “rather than being read cover to cover”, it is a handbook and a technique guide-book. In-spite of that warning I plunged in trying to gather as much as possible. Only later I realized that to make this knowledge my own, I needed to go slow and experiment with each algorithm. But blindly copying what he had done, would be of no avail.

So I set myself up with the following goals:

Goal 1: I will try to implement the algorithms in a language that I have no familiarity with.

Goal 2: I will try to stick to the original pseudo code as far as possible to increase my understanding of the technique

Goal 3: I will try to question and verify every line in the code listing provided by him, so that my implementation would be from understanding, rather than duplicating, or worse copying.

Goal 4: Have an alternate (or accompanying, if you will) source for people who come across this book, with more contextual information on implementing these algorithms. (sort of standing on the shoulders of giants!)

Goal 5: Provide a narrative of my experience, implementing these excellent algorithmic techniques.

To realize goal 1, I chose python as the language to implement the algorithms. As you can see I am a .NET guy throughout my career. So trying to implement the algorithms in python would force me to re-evaluate every line of code that I write. The problem is compounded by the fact that Jason’s original code listing is in Ruby. I had to learn just enough Ruby to understand his code and translate that in to python!. This has a dual advantage: one i get to learn a new language while applying it to a non trivial problem, two it forced me to look at each line of code from an algorithmic stand point.

To realize goal 2, I changed my implementation functions and structure to reflect the pseudo code listing that he provided. This makes it easy to follow how the pseudo code translates to implementation. In all fairness to Jason, he did a great job of it. But I wanted to take it one step further and make it more consumable. The subject matter is hard enough, I wanted to remove any further obstacles to assimilating this seminal work from Jason.

To realize goal 3, I changed some of the objective functions that he provided to others where possible. I also made some changes to some data structures and input variables to further kick the tires. I also included a test suite that exercises the algorithms. So anyone who just want to see how the algorithms behave for different inputs, can do so easily. I also put in a lot more comments in places where I ran in to difficulties. I hope it would make it more valuable to someone trying to follow along.

To realize goal 4, I want people to see how the algorithms play out in a different language. By writing this in a language that I am unfamiliar with, I hope it gives one confidence to attempt them in your language of choice. Each .py file also has a header which lays out the Name, Taxonomy, Strategy and Heuristics of the algorithm. (This is mostly from Jason’s Clever Algorithms book and algorithm template that he designed). The reason I chose to do so was, while understanding one algorithm, I wanted the code and the algorithm context at the same place. I found that very helpful in understanding and assimilating each algorithm.

To realize goal 5, I would be blogging about this. His 45 algorithms are divided in to Stochastic Algorithms, Evolutionary Algorithms, Physical Algorithms, Probabilistic Algorithms, Swarm Algorithms, Immune Algorithms and Neural Algorithms. As I finish implementing each category of algorithms, I will write a blog describing it.

I would like to sincerely thank Jason Brownlee for making available his Clever Algorithms. (By the way, Jason Brownlee did his Phd from the same university that I went to: Swinburne University Melbourne (I am from class of 2000)

Note: I tried as far as possible to write the code in a ‘pythonic’ way. Since I was learning python for this, I would like to beg forgiveness in advance from expert python developers. Any suggestions are always welcome to improve my code.

I am using Python 2.7.2 with Eclipse IDE for Java Developers Version: Helios Service Release 2. I have the PyDev ver 2.1.0xxx plugin for eclipse to work in python.

• Given two line segments p₀p₁ and p₁p₂, if we traverse from p₀ to p₂, did we make a left, right or no turn at p₁?
• Do line segments p₁p₂ and p₃p₄ intersect?

We build on what we learned in Part 1, the cross product.

Download the demo application here

Determining whether consecutive segments turn left or right

Given two line segments p₀p₁ and p₁p₂ we simply check whether the directed segment p₀p₂ is clockwise or counterclockwise relative to directed segment p₀p₁. This is similar to the solution to question 2 in Part 1. If the cross product is negative, then p₀p₂ is counterclockwise to p₀p₁. Hence we should turn left when we reach p₁ to reach p₂ while traversing p₀, p₁ and p₂. If the cross product is 0 (boundary condition) then it means all the three points are collinear (on the same line). The image below shows the three possible scenarios

Using the cross product to determine how p0p1 and p1p2 turn at p1

Determining whether two line segments intersect

Two line segments intersect if and only if either or both of the following conditions are true:

• Each line segment straddles the other line segment.
• An end point of one line segment is on the other line segment

A line segment p₁p₂ straddles another line if p₁ and p₂ are on opposite sides of the line. A boundary condition occurs when one of the end points is on the other line.

Algorithm

Calculate the orientation (using cross product!) d_i for each point of one segment w.r.t the end points of the other segment. For line segments p₁p₂ and p₃p₄ we will get d₁, d₂, d₃ and d₄.

if none of the d_i are collinear (i.e. d_i !=0) we check if d₁ and d₂ and d₃ and d₄ have opposite orientations (if one is clockwise,other should be counterclockwise). If this condition is true then we know that the line segments straddle each other and hence intersect. If it is false we can straight away say they don’t intersect.

If any d_i is collinear (boundary condition) and the corresponding p_i is on the other line segment, the lines intersect.

Shows the different cases that we encounter visually

bool Intersects(p1, p2, p3, p4)
{
    d1 = CrossProduct (p1p3, p1p2)
    d2 = CrossProduct (p1p4, p1p2)
    d3 = CrossProduct (p3p1, p3p4)
    d4 = CrossProduct (p3p2, p3p4)

    if( ((d1 > 0 and d2<0) or (d2 > 0 and d1<0)) and  ((d3 > 0 and d4<0) or (d4 > 0 and d3<0)))
    {
      then return true
    }
    else if (d1 = 0 and PointOnLine (p3, p4, p1))
    {
       then return true
    }
    else if (d2 = 0 and PointOnLine (p3, p4, p2))
    {
       then return true
    }
    else if (d3 = 0 and PointOnLine (p1, p2, p3))
    {
       then return true
    }
    else if (d4 = 0 and PointOnLine (p1, p2, p4))
    {
       then return true
    }
    else
    {
       return false
    }
}

bool PointOnLine (pi, pj, pk)
{
    if ( Min (xi, xj)<= xk <= Max (xi, xj) and Min (yi, yj)<= yk <= Max (yi, yj) )
    {
        return true
    }
    else
    {
        return false
    }
}

1. Given two points p₁ & p₂ in a co-ordinate plane, is the directed segment p₀p₁ clockwise, counterclockwise or colinear to directed segment p₀p₂? Where p₀ is the origin of the co-ordinate plane
2. Given two directed segments p₀p₁ and p₀p₂, is p₀p₁ clockwise, counterclockwise or colinear to p₀p₂ with respect to the common end point p₀?
3. Given two line segments p₀p₁ and p₁p₂, if we traverse from p₀ to p₂, did we make a left, right or no turn at p₁?
4. Do line segments p₁p₂ and p₃p₄ intersect?

Download the demo application here

Before we proceed any further a few definitions. Clockwise and Counterclockwise are intuitively easy to understand. They have the usual colloquial meaning. There is one thing to note though. A co-ordinate plane is not a watch face! Here we are determining the relative orientations of one segment w.r.t the other. Instead of trying to explain it in words, I will show you a picture (should replace a thousand words!).

The darkly shaded orientation region contains vectors counterclockwise relative to p. The lightly shaded region contains vectors clockwise to p

Two vectors are colinear if they are pointing in the same or opposite direction. In the picture all vectors that lie along the diagonal will be colinear to p.

With definitions out of the way, let us proceed to answer the initial queries that we posed. A brute force method of computing whether two line segments intersect, would be to find the line equations of the form y=mx + c where m is the slope and c the y-intercept and then compute the point of intersection, and verify whether this point lies on both lines. This involves a lot of expensive computational resources. Further this involves division which is sensitive to round-off errors depending upon the precision of the division operation. For e.g. almost parallel line segments will give incorrect results.

The algorithms given here will only use additions, multiplications, subtractions and comparisons ! No trigonometry also !
We do this by computing the cross product. This is a fundamental technique that is at the base of most CG algorithms. Let me say that again. Cross Product is a most fundamental and important technique for solving CG problems. A cross product p₁ X p₂ is defined as the determinant of the matrix comprising of x and y values of the points.

Determinant of the above matrix will be equal to (x1y2-x2y1)

If the cross product is positive then p₁ is clockwise to p₂. A boundary condition occurs if the cross product is 0. In that case p₁ and p₂ are colinear.

Proof: We use trigonometric function tan on the angle formed by the point (x,y), origin (0,0) and (x,0) for both the points. If the ratio of one tangent to other determines orientation depending upon whether the value is greater than 1 or not.

//Clockwise:
x1y2-x2y1>0

//So
x1y2>x2y1

//alternatively
(x1y2)/(x2y1)>1

//rearranging the terms
(y2/x2)/(y1/x1)>1

//Notice that tan θ is the ratio of opposite side by adjacent side
//in a right-angled triangle
//so the numerator of the above fraction is tan θ2 and
//denominator tan θ1

(tan θ2)/(tan θ1)>1

If the angle θ2 is greater than θ1
then we have proved that p1 is clockwise to p2

The above technique of using cross product makes for an easy and efficient way to answer Question 1.
To answer Question 2 just involves an extra pre-processing step before doing the cross product. To determine whether two directed segments p₀p₁ and p₀p₂, is p₀p₁ clockwise, counterclockwise or colinear to p₀p₂ with respect to the common end point p₀ we just translate the axis to use p₀ as the origin and use the cross product as above.

// To translate the origin to the common end point p0
//We let p1' (p prime) be the point where 

x1' =x1 - x0 and
y1' =y1 - y0

//We do the above for  p2 and calculate its prime: p2' 

x2' =x2 - x0 and
y2' =y2 - y0

//Then we calculate the cross product of p1' x p2' 

If it is positive then the directed segment p0p1 is clockwise to
directed segment p0p2

In Part 2 we will answer questions 3 & 4 using the techniques and principles learned here.