Back in June, someone hacked this site, and added a malware iframe.

I cleaned it up, upgraded Wordpress, and went on with my life, assuming that someone had just exploited a post.

Well, it was hacked again, the same way. Since I had upgraded Wordpress, I was more suspicious, and I delved into my website setup.

When I went to the User tab, I saw this:

Administrators(2)

(And I expected to only see one) But when I went to the adminstrators tab, I saw this:

Administrators(1)

hmmm. That’s suspicious. I looked at the page source, and sure enough, there were two entries, but the second one was buried in a morass of javascript and styles. Egads - someone has not only gained administrative access to my blog, but they have effectively prevented me from removing them via the website.

Luckily for me, I know enough about SQL to be dangerous, so I went into PhpMyAdmin and deleted all the other users except me. This worked because the only users were random bogus registrations trying to get around my spam filters.

I’ve since turned off new registrations, and I suggest everyone else do the following:

1. Check on your users - see if you have more administrators than you expect, and if so, delete them with extreme prejudice.
2. turn off new user registration

Bleah

I spent most of the last post talking about magic, but what I intended to talk about is the regular occurrence of the database getting out of sync with the domain objects in Grails.

In theory, based on using the Update directive in the database configuration, my database should be updated to match the domain objects. In my experience, however, Grails regularly fails to properly upgrade the database, and I’m forced to delete the whole thing in order to get the system sane again. This doesn’t happen all the time - generally only when I need to create a new relationship between tables or apply a new constraint (for example, allowing a column/field to be null).

Luckily, since I’m using Hypersonic SQL, this is straightforward. Shut down your grails app, and from your grails project top-level directory:

cd db
rm devDb.*      # or whatever the name of your database is

Then restart grails. It will regenerate the database (without data) with the latest constraints and structures you’ve made. You’ll need to repopulate, but there are tools for that.

In Rails, your domain model objects derive their structure and content from the database layout. In Grails, however, the database tables are created based on the fields and directives embedded in your .groovy files.

Of the two mechanisms, I find Grails’ approach to be more sensible for greenfield applications (i.e. no legacy database), and for legacy applications, it’s probably a wash, unless the database is already named in a Rails-friendly way.

I like the Grails approach because it does not violate the ‘Do No Magic’ directive.
What, pray tell, is the ‘Do No Magic’ directive?

Good question! Let’s consider a software thought experiment. You’re a novice programmer, and you’re assigned to work on an existing project. You understand the basics of software development, logging, etc, but you’re not an expert.

Your first job is to fix a bug. You have access to the source code, you know generally where the bug is, and you have log files that show the bug in action.

So you look at the source, and then look at the log files. Let’s say your source looks like this

    log.debug "About to do Step A"
    step_a()
    log.debug "Completed Step A"
    log.debug "Program Finished

And inside step_a() you see:

    def step_a() {
        log.debug "Starting Step A"
        // biz logic
        log.debug "Completing Step A"

But when you look at the log file, you see this:

1000 : About to do Step A
1001 : Completed Step A
1002 : Program Finished
1003 : Launching Thread
1005 : Activating Queue
1006 : ... other stuff...

How is this possible? Some of you may ask. My answer is - in this case - Aspects - which ‘intercepted’ step_a() and did something else instead.

But at no point in the code do you see the possibility of said interception. You have to know ahead of time that Aspects are running, in order to understand this code. To the uninitiated, this code is magic.

Rails has a similar problem - you have to look at the correct table in the database to discover how the domain objects are structured.    Again, to those who don’t know that Rails derives models from database columns, this is magic

Let’s not even speak of monkeypatching, which might more properly be called duck punching.

Hold Your Flamethrowers

To be clear: the use of database for domain objects in Rails is a relatively benign form of magic - it isn’t nearly as bad as aspect-injected  asynchronous thread launching and other such mischief.   But it is still more magical than Grails.

One of the interesting things I’ve had to work with on my Grails project is compilation.  Occasionally, I’ll get a .groovy file that seems to put Grails into an endless ‘compile-load-restart’ loop.   To discover which file is the culprit, I physically stop the server after the compilation, but before the restart, and then go into $HOME/.grails/<VERSION>/<Project_Name>/classes/  and look to see which file was most recently compiled.

This tells me a little about what’s going on, and I was able to remove the .groovy file in this case to stop the endless restarting.

One frustrating thing I’ve discovered with Grails is the way data is sent to the view pages.

In Rails, it looks a little like this:

@scenario = Secenario.new
@scenario.title = "New Scenario"
@scenario.description = "Add A Description Here"

render :view => 'create'

This snippet creates a new Scenario object, populates it, and tells Rails to render the create.rhtml file, with the @scenario object in the page’s model, helpfully known as @scenario

In Grails, it looks like this:

def scenario = new Scenario()
scenario.title = "New Scenario"
scenario.description = "Add A Description Here"

render( view: create, model: [ scenario: scenario ])

and, similarly, Grails knows to render the create.gsp file, and the scenario object is available to the page as scenario

Which isn’t terribly different, and fairly easy to use.

The problem comes in because of the way Grails uses closures to provide the action methods on the controllers (edit, create, delete, save, etc).

In Grails, you might have a action method as such:

def create = {  // closure of action method here }

Now, look at the previous Grails code. Do you see the issue? Yes - create is now overloaded, and in some cases, Grails will attempt to find a page named Controller_closure_blah_blah_blah.jsp, which is obviously completely wrong.

The fix is simple:

render( view: 'create', model: [ scenario: scenario ]) 

Basically, make sure you always use quotes for your string values, even though the Groovy language allows you to leave the quotes off. Otherwise, you’ll occasionally get frustrating and confusing results.

Interesting issue that left me scratching my head this morning…

using gsp tags, you can iterate over arrays:

   <g:each in="${myarray}">
      <p>${it.title}</p>
   </g:each>

but if you do something slightly more clever:

   <g:each in="${myarray}">
      <p><g:link action='myaction' id='${it.id}'>${it.title}</g:link></p>
   </g:each>

You’ll get null pointer exceptions.

Why?

Because the <g:link> tag creates its own instance of the special variable it inside its own ‘domain’.

Fix

add the parameter var=’myvar’ to the <g:each”> tag: <g:each in=”${myarray}” var=”myvar”>. Then, use ${myvar.id} and ${myvar.title} in your code:

   <g:each in="${myarray}" var="myvar">
      <p><g:link action='myaction' id='${myvar.id}'>${myvar.title}</g:link></p>
   </g:each>

And everything should work again.

If you’ve used Rails, you’re familiar with the flash, and all the nifty things you can do with it.

In Grails, you have flash, but you also have an entire tag library of <g:hasError>, <g:renderErrors> and so forth. None of which, as far as I can tell, care one whit about the flash. They’re all focused on domain objects, and the errors associated with processing them.

(In my opinion, this is one of the frustrations of Grails examples - the tendency to get the domain connected to the view via scaffolding, and then the author changes the subject)

But you can use the flash, in much the same way as Rails. For example, I put the following in my top-level layout template:

              <g:if test="${flash.error}">
                 <div class="errorbox">
                    ${flash.error}
                 </div>
              </g:if>

              <g:if test="${flash.message}">
                 <div class="messagebox">
                    ${flash.message}
                 </div>
              </g:if>

(note that you’ll have to define the messagebox and errorbox css classes)

Putting information into the flash is straightforward:

     flash.error = "Your error message here."

Hope this helps!

Can’t resist

Clif Bar Triathlon Start Commercial

Heheheh

My site was “partially” down for a week or two, because someone hacked a malware iframe into one of my posts, and Google flagged it.  This cascaded into Firefox 3 refusing to let me visit my site (bleh).  I was able to find the post, remove the offending iframe (and upgrade to the latest version of WordPress).

Still, more than a little frustrating to discover accidentally that my website was blocked.

8.3 miles running yesterday (1 hour, 40 minutes)
35.3 miles biking today (2 hours)

Yeah, I know I’m slow compared to all you uber athletes out there. I’m just working with what I have

An interesting comment on estimation:

1. Nobody ever does it. In fact, I don’t even know of a process to achieve this. Hollering at people who over/under estimate is not an improvement process.
2. It assumes you can make developer estimates better. More experienced developers estimate better, that I’ll take as a given, but can you accelerate this with novice/junior developers or testers?…
3. Software is NOT like mechanical engineering. It is a craft. … So our inability to accurately and precisely estimate shouldn’t be all that surprising.

Personally, I believe that these claims are false.

1. People estimate all the time - Velocity based on complexity, jellybeans, gummy bears and ideal hours are all fairly rigorous forms of estimation, when they’re done consistently.
2. There is plenty of evidence that mechanisms such as the Delphi Method do, in fact, make the general estimate far better, even when you include a mix of junior and senior developers. I use this extensively, and it has never done me harm yet. When you have five smart people discussing how hard a particular task is, you find out the different perspectives quite quickly.
3. While I agree that software is a craft of creativity, no one (that I know of) thinks that estimates have to be train-schedule accurate and/or precise. When I wear my project manager hat, I just want a general feel about how many tasks will fit into my two week iterations.

Insightful commentary on the future of spam.

He (or She) is right - Spam-management is likely to be one of the critical priorities for the future, before the entire Internet crumbles under the tragedy-of-the-commons Spam assault.

There is one “ideal world” answer, which would essentially stop spam in every possible permutation - the use of micropayments.

Specifically:

• A micropayment (by you) each time you post a comment
• A micropayment (by you) each time you link to a blog and it generates a trackback
• A micropayment (by you) to the recipient of every email you send
• A minipayment (by you) each time you sign up for an account, and then a micropayment each time you reach out to someone using that account - facebook wall messages, twitter posts, craigslist ads, /tells in MMOs, etc. The receiver will receive the micropayment.

I know, roll your eyes, micropayments are so played out. Well, I admit that they have been, and continue to be ahead of their time.

Captchas

Every other solution I’ve heard proposed involves more complex Captchas. Captchas are a hurdle, but, by definition, not an insurmountable one (because they have to be simple enough for below-average humans to pass them). Spammers continue to develop ever-increasingly-sophisticated Turing Machines, and are probably on the cutting edge of some forms of pattern recognition/AI. Each time we come up with a test, the spammers will eventually beat it. Unless you believe that computers “can’t ever” be smart enough to pass the Turing test, you have to assume that no Captcha will stand forever.

Essentially, Captchas are like building a wall in front of a horde of oncoming Mongols. They will eventually get over it. Once over it, it becomes a useless bit of architecture.

Micropayments

Micropayments, on the other hand, create an economic cost for every attempt at communication. And for most people, who receive communications about as often (roughly) as they send them, this is no big deal. If I send out 10,000 messages next year, at $0.001/message, that will cost me $10. And if I receive 10,000 messages from other people, that will reward me with $10 in credits. If I am a big talker, perhaps I spend a few dollars a year to keep my account balance up. But the point is, I don’t spend so much money that it becomes economically meaningful to me.

Spammers, on the other hand, if they send out 10 million messages, are looking at $10,000 in expenses. That’s a lot more money, and not something that can be done “lightly”.

The biggest problem in all of this is the starting-up problem. No one wants to be first website to demand micropayments, and hassle their users with an extra sign-up step. But referring back to the article - that - the generalized willingness to setup a micropayment solution - is the shape of “Web 4.0″

The Scenario

Here’s how I think it could go down - Google, Yahoo, EBay, Amazon, Microsoft and various other major players hash out a protocol for micropayment transfers. Then, they license the protocol to other organizations to implement, with the rule that every licensee has to accept and reciprocate payments with every other licensee.

My next thought is that once money (yours, mine, etc) goes into the system, it can never come out again - the micropayment universe is essentially a financial black hole - Why? To make it far less attractive as a target for hacking.

The Business Model

The Micropayment “Banks” would make money when you or I sign up our blogs to accept micropayments - a few dollars a year from a bunch of bloggers and smaller websites, a few more dollars from larger websites, etc, and you have a sustainable business model.

Accountability

The Licensees would be obliged to keep their books transparent - how much cash they received, how much they received in micropayments from others, how much they gave out in micropayments. That might even be part of the overall software specification. Someone (or many someones) can audit the books of the various Micropayment Banks, and verify that all the flows check out - that no one is claiming more “micropayment credits” than they actually have to give out. Standard financial audits would verify that the money coming in to each organization was legitimate and clean.

Commentary

This isn’t the most elegant solution in the world. Ideally there’s a solution that just involves the phrase “leave it up to the market”, but I can’t come up with one. That doesn’t mean it’s not out there…

Some feel that software products have to be essentially perfect in order to even have a ghost of a chance to succeed. People point to the iPhone and the iPod as examples of products that are practically perfect, down to the last detail, and are very successful.

Of course, one can point to a lot of counter-examples - projects that are less than perfect, but still quite successful (MySpace, Facebook, Google Mail, Google Docs, Twitter, etc). And, the huge pool of unknown projects - things that were never released because they could not achieve this desired perfection, and thus you are not even aware that they existed.

I think the gap here is the definition of “successful”. Some feel that a project is only successful if it has raving fans. Others focus on being useful to a lot of people, without worrying so much about perfection to the last detail.

I suspect (without statistical evidence) that you have the following kinds of results for projects that focus on perfection, vs. those that focus on “good enough”:

(Sorry the scale isn’t clear, the categories are “Never Released”, “Failure”, “Moderate Success” and “Huge Success”)

In reality, these scales are misleading, since the number of “Never Released” items is probably 10x as large (on both graphs). But in general, if you demand perfection, you have a slim increase in the chance that it will be a huge success, and a fairly significant increase in the chance that it will never be released.

Having said that, smooth functionality, elegant design and attention to detail are worthwhile. But they can be deceiving.

I gave up on Hani Sulemain when I realized that his modus operandi could be described in two steps:

1. Find something I don’t like
2. Announce with great certitude how clueless and incompetent everyone is who disagrees with me.

Alas, the chickens have come home to roost, I guess.

There’s a certain sense of “insecurity” in Hani’s post.  As if he’s suddenly lacking in confidence.  After all, someone fully confident of their opinions wouldn’t care what others thought.    I find that surprising in someone who was once so self-assured he could shower hate down on Google Code.   Now it’s almost like he is tired of being beaten up and just wants people to leave him alone.

Sorry Hani - this is the life you chose when you first decided to call someone else an idiot in public for supporting something  you thought was lame.

From a webpage about the book Higher Order Perl

 The other reason that the cover pictures are significant is that since my original goal in writing the programs was to impress my girlfriend, the cover pictures are therefore part of the output of the most successful Perl programs I’ve ever written. I wish all my programs achieved their design goals so spectacularly.

I tried that once - I had my wife write the last sentence of my first book for me. I thought it would be a great way to share the accomplishment with her.   Instead, she criticized my sentence structure and completely ruined the moment.  Alas

Mathias is perplexed at the animosity against JSR 308 - he says (paraphrased):

• As for generics, I gladly trade in the more complicated syntax for better static typing.

and

•  That’s not the case at all with annotations. If you like you can easily ignore the entire idea.

speaking only for myself, I’ll respond with:

• If one doesn’t like static typing to begin with, the fact that generics make it “better” is not a benefit

and

• In theory, one can ignore the entire idea.  In practice, as a developer out in the field, even if I scrupulously avoid them in my own code, it will be inevitable that over time:
  • Someone (Person A) will want to show how smart they are by implementing all sorts of too-clever-by-half annotations in their open source project or commercial project
  • Someone else (Person B) will want to show off how smart they are, by trying to create annotations that are twice as clever as Person A’s
  • Some third person (Person C), not knowing any better, or thinking they’re even smarter, will make a total mess of everything, creating bizzare new structures.
  • Lastly, I will be called in to help figure out how to fix/refactor/automate testing of this code, and it will be a complete nightmare.

This is a classic and commonplace event in software - for various reasons, a wide swath of programmers are both very smart, and fairly OCD (Obsessive-Compulsive).   They learn a new technology, and obsessively abuse it, attempting to find ways to use it for every possible problem (see Maven2).   They’re not bad people, they’re just focused on using cutting-edge technology to solve problems, and become the proverbial man with a hammer who sees nails everywhere…

And so the likely result of JSR 308 is that we will end up with open source projects that are littered with cryptic and broken annotations that will be zealously guarded and protected by the chosen few, who see nothing wrong with adding  incredibly convoluted new constructs to their program - after all, the code isn’t convoluted to them.   If mere mortals such as I struggle with comprehension, it just demonstrates how superior they are to me, no?

Groovy looks more attractive every day.

An excerpt:

        There once was a man who went to a computer trade show.  Each day as
he entered, the man told the guard at the door:

“I am a great thief, renowned for my feats of shoplifting.  Be
forewarned, for this trade show shall not escape unplundered.”

This speech disturbed the guard greatly, because there were millions
of dollars of computer equipment inside, so he watched the man
carefully.  But the man merely wandered from booth to booth, humming
quietly to himself.

When the man left, the guard took him aside and searched his clothes,
but nothing was to be found.

On the next day of the trade show, the man returned and chided the
guard saying: “I escaped with a vast booty yesterday, but today will
be even better.” So the guard watched him ever more closely, but to
no avail.

On the final day of the trade show, the guard could restrain his
curiosity no longer.  “Sir Thief,” he said, “I am so perplexed, I
cannot live in peace.  Please enlighten me.  What is it that you are
stealing?”

        The man smiled.  “I am stealing ideas,” he said.

And another:

    A master was explaining the nature of Tao to one of his novices,
“The Tao is embodied in all software — regardless of how
        insignificant,” said the master.

“Is the Tao in a hand-held calculator?” asked the novice.

“It is,” came the reply.

“Is the Tao in a video game?” continued the novice.

“It is even in a video game,” said the master.

“And is the Tao in the DOS for a personal computer?”

The master coughed and shifted his position slightly.  “The lesson is
        over for today,” he said.

Most people, myself included, can be deluded into thinking that all the great discoveries happened long ago.   That, for the most part, we have everything figured out.

I suggest you read this article about memresistors.

Fact: This technology was theorized 30+ years ago, but not demonstrated until April 30th of this year.

Fact: One of the fundamental principles of electronic circuit theory was wrong.  As  wrong as claiming that Force == Mass * Velocity, instead of Force == Mass * Acceleration.

Fact: The evidence that this principle was incorrect was here all along, but buried and shoved aside as ‘hysteresis’

What, in your lives, is being ignored because it doesn’t fit theory?   Maybe you should be focusing on what theory doesn’t explain, instead of what it does.

Oh, and as an aside - this new technology is yet another recent innovation that will make computers faster, smaller and better.  I have friends who claimed 10 years ago that we were already at the theoretical limits of computing and solar conversion efficiency and so forth.      But the world didn’t listen to them, kept innovating, and now-a-days it seems like we can’t go a month without some exciting new advancement in technology.

I haven’t seen this language before, but I’m certain as soon as we have a web framework for it, it will go mainstream.

Fair warning: The name is a swear word.

Next stop:   BF on Frails!