JBoss Amputation

23 September 2008

java jboss jbossas rails

Since JBoss AS 5 is built on top of Microcontainer, it's effectively a network of beans just doing their respective jobs. You have already probably noticed that it ships with 3 included configurations: minimal, default, and all.

Unfortunately, they're awefully far apart along the spectrum of configuration options. The minimal configuration barely gets the container running, while the all configuration includes, well, everything. For most apps, it's safe to start with the default configuration. But default to who? It's the 80% case. But the 80% of the world who gets by with default, it still probably includes way more than they need. Of course, each user needs a different subset of that 80%.

Given that the MC is managing a graph, it seems like we should be able to actually perform a solve to determine what is or is not required. Or at least get a good idea. Bonus points if someone can then twiddle conf/, deploy/ and deployers/ to tidy things up.

Anyhow, for the jboss-as-rails project, I'm kicking it old-school, and going with default. I'm jamming it into git, and hopefully with some help, we can get it pared down to what we need.

The jboss-as-rails project is simply a configuration of AS that includes only what we need, along with the jboss-rails.deployer from the jboss-rails project (notice the subtle naming difference...).

Here's a picture to see how it'll all ultimately fit together.

The plugin will offer rake tasks for managing the included AS, deploying your app. I'll also produce an AS-free version of the plugin, assuming you want to manage your own AS separately.

And poke around jboss-as-rails, see what we can rip out.

JBoss on Rails

22 September 2008

java jboss jbossas rails ruby

Tomorrow is my first real status update call with my boss, Sacha Labourey. I've been anxious to deliver something, to prove I hadn't gone completely pudding-brained during my tenure as management.

This morning, it all finally came together in a pleasing fashion, causing me to hoot and holler loud enough to scare the cats and probably some cows.

I've just pushed an ugly-but-working deployer targeting JBoss-AS 5.0.0.CR2 (the latest and greatest!)

It's not very consumable at this point, as it's just a deployer, not a nice Rails plugin with a set of Rake tasks. Heck, it doesn't even undeploy yet.

But adding the deployer to your server's deployers/ directory allows you symlink live RAILS_ROOTs into your deploy/ directory, and be running on JBoss.

Live. In-situ. Edit your controllers or views as you like, and your changes are immediately reflected in the running instance. Just like with ./script/server. It does not even have to redeploy your app. The rails framework is handling the magic reloading.

It's taken me some time to dig through the innards of JBoss-Microcontainer, and a few false starts, but I finally figured out a super simple deployment process.

I'd previously been trying to manipulate a RAILS_ROOT into a synthetic Java WAR archive, and shoe-horn things around that. But I have the freedom to go lower than that, so the jboss-rails deployer just sets up a Catalina context appropriately, without regard to WEB-INF or other non-Rails stuff. There's no need for that cruft. Likewise, I can directly control and manipulate the classpath, so the RAILS_ROOT does not even have to have any JRuby bits in it.

The example application (src/test/ballast) is a virgin rails app with ActiveRecord disabled so I don't have to deal with database-driver gems just yet.

Once deployed, a Rails app looks like pretty much any other web-app. The jboss.rails.deployment domain contains deployment objects for each rails app. And jboss.web contains all the webby bits floating around.

I need to go back and remove the dead-end code I've left in my wake, and update the tests I'd disabled while in a coding flury (bad Bob!) I plan to put together an easy-to-consume plugin gem which contains an nicely-configured AS along with the jboss-rails deployer pre-installed, along with rake tasks to start/stop AS, and deploy your app. I'd also like to give clustering a whirl, and see what we can do.

It's been an excellent 3 weeks back as an engineer.

Maven, Java and RSpec

18 September 2008

java junit maven rspec testing

Since I've been back on the job, writing Java code lately, that means I've been testing Java code lately.

After living in the land of Ruby with RSpec, thinking about JUnit did not excite me. Thankfully I found the rspec-maven-plugin, which integrates straight into the maven test process.

I like rspec because it removes a whole lot of the cruft involved in writing tests. When I return to the specs weeks later, I can still read the intent. So often, with JUnit tests named with cryptic method names, it's difficult to ascertain exactly what's intended to be tested.

The beta-4 plugin on the repository is based on jruby-1.1.2, but in SVN I've updated it to support 1.1.4. I think there's still some more improvements to be made, such as respecting maven.test.skip and getting rid of the need for JRUBY_HOME pointing to a full installation. (Note: I did not write this plugin, I've just started contributing to it).

After a little configuration of the plugin in your pom.xml...

  
<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>rspec-maven-plugin</artifactId>
  <configuration>
    <!-- jrubyHome points to the JRuby installation you wish to use (usually ${env.JRUBY_HOME}) -->
    <jrubyHome>${env.JRUBY_HOME}</jrubyHome>
    <!-- sourceDirectory references where your RSpec tests reside -->
    <sourceDirectory>${basedir}/src/test/specs</sourceDirectory>
    <!-- outputDirectory specifies where the RSpec report should be placed -->
    <outputDirectory>${basedir}/target</outputDirectory>
    <!--<skipTests>true</skipTests>-->
  </configuration>
  <executions>
    <execution>
      <id>test</id>
      <phase>test</phase>
      <goals>
        <goal>spec</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Now, just start flinging out specs in ./src/test/specs/.

This is a portion of a test against a JBoss-VFS VirtualFileHandler implementation, written in Java in WarRootHandler.java:

  
describe WarRootHandler do

  before(:each) do
    @context = RailsAppContextFactory.getInstance.createRoot( "ballast", File.dirname( __FILE__ ) + '/../ballast' )
    @root = @context.get_war_root_handler
  end

  it "should have a rails:// URL" do
    @root.to_uri.to_string.should eql( "rails://ballast/" )
  end

  it "should be resolvable through java URL handlers" do
    url = java.net.URL.new( "rails://ballast/" )
    url.to_s.should_not be_nil
  end

  it "should delegate for WEB-INF requests" do
    web_inf = @root.get_child( 'WEB-INF' )
    web_inf.should_not be_nil
    jboss_rails_yml = web_inf.get_child( 'jboss-rails.yml' )
    jboss_rails_yml.should_not be_nil
  end
end

Deployers in JBoss Microcontainer

17 September 2008

deployers java jboss microcontainer

In order to deploy a Rails app, I've had to learn the innards of Microcontainer's deployer framework. After a few wrong turns, I feel like I've finally gotten a handle on it.

While we're all used to dropping in an .ear or a .war, and might think in terms of deploying these archive formats, that's ultimately one step removed from true deployment through MC.

Within MC, when you deploy a .war or an exploded WAR directory, the first step is something recognizes that the chunk you're deploying is roughly shaped like a WAR. I'll address that phase of deployment in a future post.

Knowing that the deployment is a WAR also tells MC to look in WEB-INF/ for meta-data descriptors, such as web.xml and jboss-web.xml. This is where true deployment of components starts. Deployment runs through a series of stages, with deployers setup to match particular files and stages, doing the right things at the right time.

One of the earliest stages is the PARSE stage. A deployer can be bound to this stage to be given an early chance to match, parse, and act upon any meta-data file. For normal WAR deployment, the WebAppParsingDeployer does exactly that. There's a nice hierarchy of classes to make parsing XML descriptors such as web.xml super simple.

The WebAppParsingDeployer is the bridge from a web.xml file sitting on the filesystem or in an archive to the MetaData deployment bits. The parser reads web.xml, and produces a WebMetaData object associated with the deployment. The WebMetaData is simply a nice object-tree representing anything you can denote in web.xml.

We also might have a jboss-web.xml meta-data in our WAR, and that is parsed during the PARSE stage by the JBossWebAppParsingDeployer. This deployer, like the previous, reads the XML file and creates, in this case, a JBossWebMetaData object.

Once we've parsed these .xml files, the container has enough information to build up the classpath for the component. Some of these deployers have also thrown off or modified some ClassLoadingMetaData, which describe paths that should be added to the classpath.

As the container enters the CLASSLOADER stage of deployment, other magic occurs to actually set up the classpath.

In the end, it's the JBossWebMetaData that drives the ultimate deployment, but what if we don't have a jboss-web.xml? That's where the MergedJBossWebMetaDataDeployer comes in. It looks for a WebMetaData, and a JBossWebMetaData if one has been parsed, and merges them into a singular JBossWebMetaData. I think it also mixes in any defaults that you have set for server-wide settings.

Additionally, as jboss-web.xml is parsed by JBossWebAppParsingDeployer, it will perform the merge itself. Additionally, magic is occuring to merge any annotation-based meta-data.

I'm a little fuzzy on the ins and outs of the CLASSLOADER stage at this point, but magic occurs there.

And our app still isn't deployed yet. But we're getting there.

Finally, we enter the REAL stage of deployment, which fittingly-enough, is where the actual deployment occurs. Hooray!

Our TomcatDeployer is hanging out there, waiting for JBossWebMetaData objects to appear. When it sees one, it goes to work setting up information for Tomcat to deploy a web-app. It configures everything in Tomcat from the information other deployers figured out from web.xml and jboss-web.xml and embodied in the MetaData.

It jams it into Tomcat, hits the big red "go" button, and port 8080 is serving you web-app.

Finally.

In general, to deploy in AS5 using Microcontainer, you need some MetaData bits, and perhaps a bag of files/classes/resources. Nothing says they have to be bundled into a .war, or include some j2ee XML deployment descriptor. If you have other magical ways of bundling MetaData and resources, you're good to go.

Of course, Ales or Adrian may tell me I'm completely wrong. That's always a possibility. In fact, I'm sure I've got some things wrong, in reverse order, and otherwise mixed-up.

Here's a picture for you, though.

Update:

While discussing this on the Microcontainer user's forum, I discovered that there are indeed several errors and inconsistencies in the above.

Update #2:

New image, slightly new text to match the image better. Comments and clarifications still welcomed.

F3 is a beautiful thing

12 September 2008

comprehension eclipse java vi

Code comprehension. It's important when you jump into someone else's code.

Eclipse makes it easy, with F3, command-T, and shift-command-G.

Very quickly, you can jump through a maze of classes and interfaces, diving into details or seeing the higher hierarchy. I forgot how nice Eclipse can be. Particularly if you've got the viPlugin.

72 open files (and bless the JBossAS guys for including Eclipse projects and source jars).

JGroups impersonating memcached

04 September 2008

java jboss jgroups memcached

I woke up and noticed that a memcached mode was announced by Bela Ban, the fantastically Swiss man who jogs waaay too much.

There's a lot of things in this world that can take advantage of memcached. Personally, I find this quite interesting, considering that memcached tends to be the cache-of-choice for lots of non-Java languages. For instance, fragment caching and model caching in Rails supports talking to memcached.

Or now, JGroups.

Another JBoss GitHub repository mirror

30 August 2008

git github java jboss jbossorg mirror svn

For those of you playing along at home, I've added jboss-deployers to my GitHub mirror set. Like the others, the 'vendor' branch is the one you want.

I'm adding JBoss projects to my mirror set as I trip across the need to browse their source. If there's a JBoss project you'd like to see mirrored out of SVN, drop the URL to the trunk of the SVN repository in a comment on this post, and I'll start slurping it.

And now, something slightly different

28 August 2008

java jboss jbossorg job jruby ruby

Back in May, I was a manager.

I feebly attempted to direct 8 great guys and gals to further the goals of JBoss.org. After the Codehaus, you'd think I'd be able to help build an opensource community with fun and flair. But I came to realize that it's hard to build a community as an active effort. Instead, I think community develops as a by-product of a useful and well-run project. And that's under the control of the project leaders and contributors, not necessarily some external third party.

Back in May, I gave up being a manger.

Now, the day after Labor Day, fittingly enough, I'll be jumping back into the world of JBoss. But not as a manager. When I was burned out and felt like resigning, Mark Proctor and Sacha Labourey instead talked me into taking a sabbatical. And I'm truly grateful to them. Now, after unwinding for a few months, I asked to rejoin the team as an engineer. Through Sacha's patience and budget manipulation, I'm once again excited to go to work. I think JBoss should definitely be held up as a company that takes care of its people. They could've easily given me the boot, but instead they've been extremely kind and accommodating.

So, what will I be doing?

After talking to Java developers and Rubyists alike, my first goals are to look at Rails as just-another-way to write J2EE apps (or "JEE" I reckon, these days...). Yes, I know about (and plan to use) things like Warbler and JRuby-Rack. Both are good things.

But I also have full control of the deployment environment, to build a stack to make it happier than "build and deploy a WAR".

Through the miracle of AS5 built on JBossMicrocontainer, along with the awesome VFS bits, it should be possible to deploy a Rails app in-situ, right from your working directory. There should be no reason to have to build a WAR while you're hacking a rails app. And deployment to a server should still involve capistrano (in my opinion). Stick to the Rails way of doing things, but make it Java under the covers.

Various blog posts have shown Rails apps on Glassfish in 12, 10, or 5 steps. My goal is to get it down to 1 step. And you should magically be able to pick up and use all the wonderful JEE bits that maps to the Rails functionality the Railers of the world enjoy, without having to be aware of the JEE bits.

Speaking with Mark Newton (the guy who runs JBoss.org now), it seems sensible to view Rails as simply yet-another-programming-model for writing Java apps. The idea is to avoid leaky abstractions, so we're not having to write some psuedo RubyJava application.

Once we've got that base covered, then we can make fun and exciting Ruby bindings to all the powerful JBoss tools, such as Drools, ESB, Cache or MQ.

I expect to have a bit of fun with this. More fun than being a manager, certainly.

GitHub Mirrors for some JBoss Projects

21 August 2008

git github java jboss jruby svn

In addition to the previously-mentioned JRuby mirror from Codehaus SVN to GitHub, I'm now also mirroring:

All are trunk-only mirrors, not picking up branches or tags. Since the JBoss repository path has about 77,000 subversion revisions, and at one point held any and all JBoss software ever written, I have not mirrored it in its entirety. Instead, I've only grabbed http://anonsvn.jboss.org/repos/jbossas/trunk back to revision 77,200. It'll mirror going forward, but the github repository does not include any ancient history.

For those of you playing along at home, the way to fetch just a cauterized "tip" from SVN to a git repository is to mirror as before, but for the initial "git svn fetch" command, add a SVN-style revision range

git svn fetch -r77200:HEAD

For me, at least, trying to fetch the tip revision for the directory resulted in failure. Going back a few revisions, and using a range that includes HEAD worked much better. Then just push to GitHub has normal, and start your rebase/push cronjob.

The JBoss projects are updated from SVN every 15 minutes. But we're updating from the anonymous SVN repository at JBoss, which itself is delayed from the developer repository by some amount of time. So, ultimately, the GitHub mirror should be mostly up-to-date, but could lag behind actual developer commits by up to and hour, I reckon.

If you're wanting to track these repositories using my git mirror, only track the vendor branch. I make no claims about the stability or sanity of the 'master' ref at any point in time. I will make sure 'vendor' exactly matches the Subversion history, though.

Mirroring SVN repository to GitHub

20 August 2008

git java jruby subversion

So, I'm gearing up to work on some Java+Ruby (via JRuby) stuff. The Java world still seems fairly entrenched in the cult of Subversion, while the Rubyists have gone with Git lately.

I'm still wrapping my mind around Git, but with GitHub, it's fairly easy and straight-forward. I paid my $7 for the micro account, to give me room to screw around.

There's quite a few posts about mirroring SVN to a Git repository, but I feel the need to add my own, of course.

My goal is mirror the trunk of the JRuby project from Codehaus SVN to my account on GitHub. By doing this, I can track the trunk development, and also work on my own patches.

I started by creating an empty repository on my GitHub account, called 'jruby'.

http://github.com/bobmcwhirter/jruby/tree/master

Now, over on my always-on, Contegix-powered server, I create a brand new local git repository, also called jruby.

mkdir jruby cd jruby git init

Next I use 'git svn init' to setup the SVN repository as a remote code source to track. Using the -T switch points git to the trunk, and ignores branches and tags, which is fine for my purposes.

git svn init -T http://svn.codehaus.org/jruby/trunk/jruby/

That does not pull any code, but it lets my local working tree know that I'm going to be pulling from an SVN repository at some point. This setup only occurs in your local repository, and does not seem to ever get pushed to GitHub once we get to that point.

So, now we do the initial pull. Once again, this is on my always-on, Contegix-powered server, not my local laptop. I'm doing this on a server because towards the end, we'll be setting up a cronjob to accomplish it all.

git svn fetch

It'll think for a while, it'll slurp down the SVN revision history, it'll stop and ponder occasionally, and eventually, it'll be done. Woo-hoo! Our local working tree is now up-to-date with the subversion HEAD as of that moment.

To reduce disk-space used by your local repository, go ahead and run the garbage collector

git gc

On my system, that reduced the space from over 600mb to under 70mb.

Now, that's great, but it's still just on my local repository. Time to push it to GitHub. We're not going to follow their directions exactly, since this will ultimately be a cronjob and needs to use ssh. And I'm slightly paranoid about my ssh keys.

So, the first thing I do is create another keypair, for used only by my mirroring process, and only for pushing changes to github. It has no passphrase. This allows me to keep my top-secret keys off my shared, always-on server. If these keys are compromised, all an attacker can use them for is to push changes to GitHub. Which, being revision-control, is more annoying than dangerous. (Hooray for "git reset").

ssh-keygen -t dsa -f .ssh/id_dsa_github_mirroring

Next, I edit my .ssh/config to add a "fake host" so that ssh connections invoked by git will use this new key.

As with all previous bits, this is still on my always-on server, not my local laptop.

Host githubmirror User git Hostname github.com IdentityFile /home/bob/.ssh/id_dsa_github_mirroring

This will cause any invocation of "ssh githubmirror" into "ssh git@github.com -i .ssh/id_dsa_github_mirroring".

I then installed id_dsa_github_mirroring.pub into my GitHub account.

Now, GitHub's instructions say to run this command to add the GitHub repository as a remote named "origin"

git remote add origin git@github.com:bobmcwhirter/jruby.git

Instead, we teak it to use the "fake host" we added to .ssh/config

git remote add origin git@githubmirror:bobmcwhirter/jruby.git

We're almost done, I promise.

Next, we need to do the first push from my server up to GitHub. We first push to the 'master' branch, since the repo really wants to have a master branch.

git push origin master

Now, GitHub doesn't allow you to fork a repository you own, and since this mirror is owned by me, where can I do my own hacks and patches? The 'master' branch of course. But I still want an unmolested, straight-from-subversion mirror. So, I create a 'vendor' branch in my workspace. It's initialized to match 'master' exactly.

git checkout -b vendor

Now, I push that to GitHub, too.

git push origin vendor

Awesome. I now have two branches, identical at the moment, called "vendor" and "master".

Now, as far as I can tell, all the Subversion setup that we did only lives in the local repository on my always-on server. Anyone who clones from the GitHub repository will not have that stuff. They can of course do a 'git svn init' themselves, to add it to their local repository. But it doesn't flow through GitHub.

But that's fine, since I've been doing this on my always-on server anyhow. My workspace is sitting in the 'vendor' branch that's tracking the vendor branch from github.

I can pull the latest changes from Subversion by typing

git svn rebase

The 'rebase' command is neat, in that any changes that exist in the git repository are floated to be applied to whatever the latest HEAD is. But since I'm only concerned with a one-way SVN-to-Git mirror, there will never be any changes to float, and this will just tack on subsequent SVN commits as Git commits onto the 'vendor' branch. It'll leave the 'master' branch un-touched.

After rebasing, you gotta push the 'vendor' branch up to GitHub.

git push origin vendor

Now, type that every 15 minutes, and your 'vendor' branch will stay mostly up-to-date.

Or use cron.

I've cronned a script that fires every 15 minutes

#!/bin/sh cd /home/bob/github-svn-mirrors/$1 git svn rebase git push origin vendor

It's run with the repository name as the first (and only) argument

*/15 * * * * /home/bob/github-svn-mirrors/bin/mirror jruby

Now, over on my laptop, finally, I can clone the repository, work on topic branches, push to master and have my own controlled environment and fork, while knowing the 'vendor' branch reflects the pure SVN state which I can also pull into my hackings as-desired.

When I submit a patch, if it ultimately floats back to me through the vendor branch, git is supposedly smart enough to realize that the same changes have arrived in my 'master' (assuming it's applied verbatim) and keep things nice and tidy. Else, I can force a merge, trampling my half-assed patch with the official JRuby code.