Be careful with magical code
Let’s talk about programming magic. It seems hip to use the most powerful language or framework and power often goes hand in hand with magic. Many kinds of magic are available, especially in languages which are renown for their magic, like Ruby. But there is a lot of magic even in Java:
- Class loading
- Garbage collection
- Terracotta (which is still a wonderful solution to the coherence problem)
- Remote procedure (RMI) or webservice calls
- AOP
- ORMs and in particular Hibernate
I’ve been a long fan of ORMs – especially Hibernate – but I’m no longer, so I wrote “Orms are a thing of the past”. Particulary because of their magic. The same goes with AOP. I’ve biten by Spring AOP, but it could be any AOP, with the more powerful like AspectJ being more dangerous. Some methods changed, Eclipse wasn’t clever enough, pointcuts didn’t find the right methods anymore and the system was broken in production.
Leaky abstractions manifest themselves often because of magic. Early Java examples of magic which can go wrong are garbage collection (GC) and RMI. Garbage collection works like a charm, up until it doesn’t work anymore or a full GC run stops your application for several minutes. RMI, another early one, breaks in ways you do not anticipate:
calc.add(2,4)
Reading this code, one would assume very little can go wrong. But if it’s a RMI call it can throw some nasty exceptions at you. As I’ve commented in my post “ORMs are a thing of the past”:
But association managment with ORMs has the same problem as RMI: It seems to be transparent, but it is not (lazy exceptions and remote exceptions).
What is programming magic? Is it Vodoo? Arthur C.Clarkes third law says:
Any sufficiently advanced technology is indistinguishable from magic.
which makes magic something you do not understand. Any sufficiently advanced programming language or framework. As a sidenote, programming magic is bordering and related to cargo cult programming.
Why is magic even used?
Magic is good. It helps developers safe time, work with higher abstractions, write less code, read less code, have more aesthetically pleasing code and it makes you feel powerful. And it seldom does bite the one who wrote the code. Problems often arise in maintanance, with developer turnover and new developers needing to understand the code. Magic most often bites operations, and because developers and operations often do not talk, it seems like a SEP.
As I’m working more in operations in the last years, I have been biten by Java code. I have no experience with Rails ops – someone can fill me in? – but assume it is the same in Rails and PHP as it is in Java operations.
Why does magic bite you?
Because
Magic is non-linear
Repeat: Magic is non-linear. It works for a wide range, then suddenly breaks.
- High Load under special circumstances
- Many, many more queries than you thought of
- Locking in unpleasent ways
- Exceptions you didn’t think of
And because you do not understand the magic it’s hard to know the breaking point. Assume
customer.getAddresses()
which differs a lot when it’s proxied with an ORM, compared to a local method call. Perhaps it produces n+1 queries, it might be lazy, might take a lot of time under load or might have non-linear and unknown caching behaviour.
What you can do
You can and should use magic, but need to balance with maintanence and operations:
- Know where it’s happening
- Confine it to defined points
- Really understand what’s going on
“Confine” today is a very strong concept to me, and most of the code I write or review confines magic to clearly understood place. I’ve also learned to write a small anti-corruption layer around all external frameworks, but this is better explained in another post. Taking the Hibernate example – and I will get flamed for this – you might use:
dao.getAddresses(customer)
instead of
customer.getAddresses()
With a simple implementation this looks like:
class HibernateCustomerDao extends CustomerDao {
def getAdresses(c:Customer) = c.getAdresses
}
Why is this better? A developer knows it’s non-linear when he sees dao.getAddresses(customer) compared to customer.getAddresses(). And sooner or later you will need to optimize performance or change caching. Easy to swap with JDBCTemplate for performance reasons, to add caching etc.
Other kinds of magic?
There some more kinds of magic you need to beware of. All the time there is new magic coiming up. Is Project Lombock magic? Probably not. Is type inference in Scala magic? I’m not sure yet, there is some discussion. Are implicits in Scala magic? I’m pretty sure they qualify. Beware of magic. Use it to your advantage, know it, confine it and really understand what’s going on.
You can leave a Reply here. Of course, you should follow me on twitter here.
There never is magic, only complexity. Computers are deterministic.