Four short links: 15 May 2017
Formal Systems, Deep Learning, Assembly Games, and Logs vs. Metrics
- Form and Content in Computer Science (Marvin Minsky) — Minsky’s 1970 ACM Turing Lecture. Let us consider a more elementary, but still puzzling, trade-off, that between addition and multiplication. How many multiplications does it take to evaluate the 3 X 3 determinant? If we write out the expansion as six trinomials, we need 12 multiplications. If we collect factors, using the distributive law, this reduces to nine. What is the minimum number, and how does one prove it, in this and in the n X n case? The important point is not that we need the answer. It is that we do not know how to tell or prove that proposed answers are correct! The interesting work currently being done in formal systems has a long heritage, but struggled for attention and interest in researchers for a long time.
- Questions & Intuition for Tackling Deep Learning Problems — a great list. Never mind a neural network; can a human with no prior knowledge, educated on nothing but a diet of your training data set, solve the problem? Is your network looking at your data through the right lens? Is your network learning the quirks in your training data set, or is it learning to solve the problem at hand? Does your network have siblings that can give it a leg-up (through pre-trained weights)? Is your network incapable or just lazy? If it’s the latter, how do you force it to learn?
- Computer Games that Make Assembly Language Fun (IEEE Spectrum) — three polished games that do a surprisingly good job of making coding in assembly language fun. To be clear, none of these titles involve writing assembly for real hardware. They all use virtual systems with minimal instruction sets. Still, they do capture the essence of assembly coding, with complex behaviors squeezed out of simple commands.
- Logs vs. Metrics — difference between logs and metrics is huge. A log is an immutable record of discrete events that happened over time while metrics are a set of numbers that give information about a particular process or activity usually recorded over time to form a time series. I loved the RED method: “request rate, error rate, and duration of requests to tell you how busy your service is, whether there are any errors in it, and what its latency is.”
