We already know what’s topological sort of a directed acyclic graph. So why do we need a revision of this algorithm? First of all I never mentioned its complexity, thus to understand why we do need a revision let’s get again on the algorithm.

We have a directed acyclic graph (DAG). There are no cycles so we must go for some kind of order putting all the vertices of the graph in such an order, that if there’s a directed edge (u, v), u must precede v in that order.

The process of putting all the vertices of the DAG in such an order is called topological sorting. It’s commonly used in task scheduling or while finding the shortest paths in a DAG.

The algorithm itself is pretty simple to understand and code. We must start from the vertex (vertices) that don’t have predecessors.

We put them in our sorted list in random order. Since they don’t depend on each other we can assume they are equally sorted already. Indeed thinking of a task schedule if there are tasks that don’t have predecessors (they don’t depend on other tasks before them) and that don’t depend on each other we can put them in random order (and execute them in random order).

Once we have the vertices with no predecessors we must remove the edges starting from them. Then – go again with the vertices with no predecessors.

It’s as simple as that, so why do we need a revision of this algorithm? Well, basically because of its efficiency.

Overview

As we know most of the graph algorithms depend on the way the graph is represented in our application. We consider as the two main representations the adjacency matrix …

… and adjacency lists.

Let’s first take a look of some of the main approaches to get the topologically sorted list at the end of the algorithm.

What can we do in order to find the vertices with no predecessors? We can only scan the entire list of vertices.

Adjacency Matrix

In case we’re using adjacency matrix we need|V|^2 space to store the graph. To find the vertices with no predecessors we have to scan the entire graph, which will cost us O(|V|^2) time. And we’ll have to do that |V| times. This will be |V|^3 time consuming algorithm and for dense graphs this will be quite an ineffective algorithm.

Adjacency Lists

What about the adjacency list? There we need |E| space to store a directed graph. How fast can we find a node with no predecessor? Practically we’ll need O(|E|) time. Thus in the worst case we have again O(|V|^2) time consuming programs.

So what can be done in order to optimize this algorithm?

Practically we can start by picking up a random vertex and “go back” until we get a node with no predecessors. This approach can be very effective yet also very ineffective. First of all if we have to scan all the way back to a node with no predecessors this will cost us |V| time, but if we stuck on a node that don’t have a preceding node then we’ll have a constant speed.

This means that we can modify the algorithm a bit in order to improve a lot the algorithm. We just need to store both incoming and outgoing edges and slightly modify the adjacency lists.

What’s the algorithm now?

First we easily find the nodes with no predecessors. Then, using a queue, we can keep the nodes with no predecessors and on each dequeue we can remove the edges from the node to all other nodes.

Pseudo Code

1. Represent the graph with two lists on each vertex (incoming edges and outgoing edges)
2. Make an empty queue Q;
3. Make an empty topologically sorted list T;
4. Push all items with no predecessors in Q;
5. While Q is not empty
   a. Dequeue from Q into u;
   b. Push u in T;
   c. Remove all outgoing edges from u;
6. Return T;

This approach will give us a better performance than the “brute force” approach. The running time complexity is O(|V| + |E|). The problem is that we need additional space and an operational queue, but this approach is a perfect example of how by using additional space you can get a better performing algorithm.

Although this post is supposed to be about algorithms I’ll cover more on graphs and their computer representation. I consider this very important, because there are lots of problems solved by using graphs and it is important to understand different types of representation.

First of all let’s try to explain what is a graph?

A graph is a specific data structure known in the computer science, that is often used to give a model of different kind of problems where a set of objects relate to each other in some way . For instance, trees are mainly used in order to represent a well-structured hierarchy, but that isn’t enough when modeling objects of the same type. Their relation isn’t always hierarchical! A typical example of graph is a geo map, where we have cities and the roads connecting them. In fact most of the problems solved with graphs relate to finding the shortest or longest path.

Although this is one very typical example actually a huge set of problems is can be solved by using graphs.

As shown on the image above the graph is a “more complex” data structure than the ordinary tree. Thus a graph supports cycles, while the tree doesn’t. In the other hand the nodes of a tree are defined by their parents and children, while in a graph that isn’t true.

In this case each graph is defined by its edges and its vertices. In most of the cases, in order to model and solve our problem, we can assume that the vertices are consecutive numbers starting from (1, or 0 in case of 0 based arrays, as we will see later).

As we see each tree is a graph, but not every graph is a tree.

In first place we must now that graphs can be divided in several categories.

They can be undirected and directed. An undirected graph means that in case there is an edge between the nodes i and j we shell assume that there is a path from i to j, as well as from j to i. In the case of directed graph, we’ll assume that if (i,j) exists there only path from node i to node j and there’s no path between j and i.

In this example we assume that all the edges are the same, which in practice isn’t always true. Taking a look back to the example of cities and roads, we know that the roads between different cities are different. In many cases their length in kilometers or miles are defining the algorithm (for instance longest/shortest path). To model this we can use weighted graphs, where each edge is associated with a weight. Note that, in the example below, the weight can be even a negative number. Of course in the example of cities and road that can’t be true, because we can’t have negative distance, but in some cases (let’s say where the path saves us some money) we can have negative values.

To complete the whole image, let’s give another example which will make the difference between graphs and trees even bigger. Graphs can be connected and disconnected. This means that the graph is constructed out of two or more sub-graphs without a path between these components. You can think of a disconnected graph as for the roads of the UK and France, since they aren’t connected by land.

Overview

We know what a graph is in general. However we need an appropriate way to represent them in our programs.

There are many type of representation, which can be very useful in some cases and very useless in others. Two of the mostly used types of representation are the adjacency matrix and the adjacency list.

Adjacency Matrix

In the first case we store a matrix (two-dimensional array) with size NxN, where N is the number of vertices. This means that for each edge between the vertices i and j we have the value of 1 (A[i][j] = 1), and 0 otherwise.

In case of directed graph, we can use 1 for the edge (i,j) and -1 for (j,i) in case the edge is directed from i to j.

For a weighted directed graph we can put the weights instead of 1s.

Adjacency Lists

Another useful representation of graphs are the adjacency lists. In this case for each vertex we store a linked lists consisting of all of his successors.

Although these two ways are the mostly used, there are also some other type of representations. Such a useful representation is storing only the connectivity between two vertices i and j only if there’s a path between them. Of course this can help us answer the question “is there a path between i and j” in O(1), but unfortunately we lose the information about the graph and we can’t build it again out of this representation.

Complexity

Most of the basic operations in a graph are:

1. Adding an edge;
2. Deleting an edge;
3. Answering the question “is there an edge between i and j”;
4. Finding the successors of a given vertex;
5. Finding (if exists) a path between two vertices;

Thus depending on the representation these operations can have different complexities. In case that we’re using adjacency matrix we have:

1. Adding an edge – O(1);
2. Deleting an edge – O(1);
3. Answering the question “is there an edge between i and j” – O(1);
4. Finding the successors of a given vertex – O(n);
5. Finding (if exists) a path between two vertices – O(n²);

While for an adjacency list we can have:

1. Adding an edge – O(log(n));
2. Deleting an edge – O(log(n));
3. Answering the question “is there an edge between i and j” – O(log(n));
4. Finding the successors of a given vertex – O(k), where “k” is the length of the lists containing the successors of i;
5. Finding (if exists) a path between two vertices – O(n+m) – where m <= n;

We now see that depending of the representation of the graph we can have different complexities for the same operations. This is very important while trying to solve a problem and can be crucial while chosing the algorithm.