Ever since I saw Andrew Gelman’s post on visually-weighted regression plots I knew I wanted to try something similar in one of my plots. What I like the most about those watercolors is the sense of isolation and simplicity they provide without over-burdening the reader.
Yesterday, I was working on the convergence plot for the Duplex algorithm. To show how fast Duplex optimizes the objective function, I ran Duplex for 100 executions and each execution for 100 iteration. In each iteration, I measured the distance from the current sample and the optimum value. In this particular example, we are optimizing the transistor sizing for an io-circuit, but Duplex can be applied to any nonlinear optimization problem.
The problem is when I plot all of the convergence plots together, it gets very messy. There are lots of data in this plot, but it is very hard to see because it is very cluttered. 
An easy way to simplify that plot is to draw expected convergence rate, i.e. the average error w.r.t the iteration number. Drawing the expected value is necessary to convey my message (Duplex converges very fast), but it is not sufficient, because it is abstracting too much information. Specifically, because the choice of the initial state is random, there is a lot of variance in convergence plots, which should be somehow emphasized.
So I adopted the Visually-weighted graphs. For VW-plots, I divided the Y-axis into bins, and whenever a plot crossed a bin, I would increment the value of that bin. In the end, I had to apply multiple low-pass filter over the plot to smooth everything out and emphasize the coloring of the graph. The MATLAB code for generating the visually-weighted plots is committed to Github.
In comparison to the single expected value, these plots provide more information in a visually-appealing and clean-simplified package. Specifically, I can easily show that the variance in convergence error is high (at the beginning, due to initial state being random), but quickly converges to 0 after 50-60 iterations.
Leave A Comment