This year after JavaOne, my wife and I took a holiday. We hired a car and drove round California and visited some of the amazing National Parks the USA has to offer. I was under orders not to speak about or mention the conference - I was supposed to be switching off and relaxing. Well, at Sequoia, we took a tour round the amazing (and badly named) Crystal Caves. While I was down there it seemed to me that the events of the hectic week just passed came together somewhat. Crystalised you might say.
It had seemed to me that on the surface, there wasn't a lot new out there. Most of the sessions I'd been had told me a lot, but its wasn't exciting and new. But then I realised that there was a more subtle trend - lots of things had instead been re-imagined: Glassfish 3 was now OSGi based and modular. Then there was a better Spring (which now claimed to be a better EJB 3.0 container) and also a new Guice (which claimed to be a better Spring). In other areas, there were calls to re centre the human and re look at our mental models behind things we take for granted - Kathy Sierra was a surprise guest at "How to Write the Next Great Java Book" and implored us to consider the audience's psychology when addressing them. In other areas Hudson was Continuous Inegration with a human face and Mylyn tried to challenge IDE information overload and allow you to code "at the speed of thought". Even the Aussies at Atlassian implored us to put the fun back into our builds.
The Re-imagining continued. Applets were back with a vengance breaking the browser / desktop boundary and gaining a spontaneous standing ovation for the team when they came on stage and new JVM languages were now accepted as first class citizens: we took the Red Pill with Groovy, heard how JRuby is being accepted in the enterprise from Ola Bini, and Scala came out of the academic shadows with a large crowd in attendance for Martin Odersky's latest update. Finally JavaFX Script continued to gain momentum, being discussed from a language design perspective by its creator, Chris Oliver.
And there it was. Just as my thoughts had crystallised, so it seems has the Java Ecosystem. Re crystallised in fact, from the same basic substrate - the JVM - but under changing conditions - dynamic languages, harder problems, new communities. It's still an interesting time to be invoved in the Java world. Just don't expect to be writing just public static void main(String[] args) for ever!
Thursday, May 22, 2008
Monday, May 19, 2008
CommunityOne 2008 - Open Social For Your Website
Dave Johnson and James Wood.
History: blogs, wikis, feeds and friends. All were websites.
- Enterprise vendors and portals are adding support for this kind of web site.
- Web 1.0 was hard. FTP.
- Then came wikis and blogs. All had feeds and RESTful Apis.
- Then the web became social. Friends and shareing.
- 2002-3 friendster. Myspace. Facebook. The demographic expands.
- Facebook launch their API. Become a platfom and social graph. Plugins to my web site an beyond
- Then google "answered" with Open Social.
What is a social? Share profile and activities and control the audience for this. Find friends, get a date , share a common endeavor.
As a platform. You don't build the apps. You let others do it on you.
Then rich profile info went to work. For better personalisation , sharing and group collaboration. Big vendors build this in. The social graph will converge with ID management
What is Google's Open social? Its a Javascript API built on top of google gadgets and a REST API in top of an extended atom publishing protocol ( but JSON will be added)
Open social api: It models people but you can't write to it. They can't change your data.
It also has activities: title, date, media file. Apps can create activities.
Persistence: you can store name value pairs.
The REST API mirrors JS API above.
There is an open social namespace to get details of supported functions.
Currently .v.0.7
Grouping is not done yet
Apache Shindig is the reference implementation
Plus the open social foundation is there but that's it
But there is still the silo problem. Open Social doesn't solve it. But there are some reasons to have done silos.
Linking/federation is possible (with FOAF or XHTML friends API) but there is no privacy ( though there is work to integrate FOAF with OpenID)
Is it open? Spec now out in December. Yes it is now.
Apache shindig: Javascript
Its a Java based Google gaget server.
What's missing?: It has no implementation of a social graph. You must use your one existing. You need to provide your own UI elements to edit profiles.
How do I use It?: write a facebook app? Whitelabel? Use apache shindig?
But my site didn't have an existing social graph!
Project social site:
- Build wigets and web services.
- Web services with full rest supper
- Socialsite adds a profile widget.
- Addresses UI piece with widgets. Profile, friends, groups,...
- Addresses editing, joining, accepting and managing
- It extends open social.
Value add: it has a comprehensive social graph.
- Easy to use widgets for java and ruby apps.
- Gadget management features
- ID management features with tie in to your ID management solution.
JMaki is the widget wrapper ( Netbeans and Nclipse support)
- Drag and drop to create an application
- Extended mediawiki example
- Wrote a mediawiki extension so that as a user you add a single tag to add the widgets to my profile page.
- Also done for the roller blog server as new templates.
Links:
http://socialsite.dev.java.net
http://code.google.com/apis/opensocial
History: blogs, wikis, feeds and friends. All were websites.
- Enterprise vendors and portals are adding support for this kind of web site.
- Web 1.0 was hard. FTP.
- Then came wikis and blogs. All had feeds and RESTful Apis.
- Then the web became social. Friends and shareing.
- 2002-3 friendster. Myspace. Facebook. The demographic expands.
- Facebook launch their API. Become a platfom and social graph. Plugins to my web site an beyond
- Then google "answered" with Open Social.
What is a social? Share profile and activities and control the audience for this. Find friends, get a date , share a common endeavor.
As a platform. You don't build the apps. You let others do it on you.
Then rich profile info went to work. For better personalisation , sharing and group collaboration. Big vendors build this in. The social graph will converge with ID management
What is Google's Open social? Its a Javascript API built on top of google gadgets and a REST API in top of an extended atom publishing protocol ( but JSON will be added)
Open social api: It models people but you can't write to it. They can't change your data.
It also has activities: title, date, media file. Apps can create activities.
Persistence: you can store name value pairs.
The REST API mirrors JS API above.
There is an open social namespace to get details of supported functions.
Currently .v.0.7
Grouping is not done yet
Apache Shindig is the reference implementation
Plus the open social foundation is there but that's it
But there is still the silo problem. Open Social doesn't solve it. But there are some reasons to have done silos.
Linking/federation is possible (with FOAF or XHTML friends API) but there is no privacy ( though there is work to integrate FOAF with OpenID)
Is it open? Spec now out in December. Yes it is now.
Apache shindig: Javascript
Its a Java based Google gaget server.
What's missing?: It has no implementation of a social graph. You must use your one existing. You need to provide your own UI elements to edit profiles.
How do I use It?: write a facebook app? Whitelabel? Use apache shindig?
But my site didn't have an existing social graph!
Project social site:
- Build wigets and web services.
- Web services with full rest supper
- Socialsite adds a profile widget.
- Addresses UI piece with widgets. Profile, friends, groups,...
- Addresses editing, joining, accepting and managing
- It extends open social.
Value add: it has a comprehensive social graph.
- Easy to use widgets for java and ruby apps.
- Gadget management features
- ID management features with tie in to your ID management solution.
JMaki is the widget wrapper ( Netbeans and Nclipse support)
- Drag and drop to create an application
- Extended mediawiki example
- Wrote a mediawiki extension so that as a user you add a single tag to add the widgets to my profile page.
- Also done for the roller blog server as new templates.
Links:
http://socialsite.dev.java.net
http://code.google.com/apis/opensocial
CommunityOne 2008 - The Road to Babel: Bringing the JVM Language Implementers Together
Charles Nutter, SunMicrosystems
Language survey: 200+ lands
Nolanf is inpossible fix are most diff, dynamic are hard to optimise. No Lang is too weird
Functional Languages:
clojure:
Lisp dialect but easier to read and write.
- Released every 3-6 weeks. Getting exposure now.
Kawa: Implementation of scheme. One of oldest compilers.
- Currently v1.9.1. Multiple projects using it. Open secret.
Ocaml-java: Very expressive. Functional, imperative and OO styles, need to set up external files to bind to java calsses
- 1.0 out in beta now. 2.x branch comping to focus on speed.
Scala: OO, ( classes, traits and mixins)
Statically typed. Nice looking DSLs. Fully interoperable with Java
- 2.7.0 is final. 5000 downloads / month. The Lift web framework is gaining traction.
Groovy: many features from Python and Ruby. Always JVM Lang. Has closures, mutable classes with a very java-like syntax. Has many libraries built in.
- 1.5.6 is current. 1.6 is out in beta. (Perf improvements - 500> in some cases and tidied bytecode)
JRuby: Ruby on the JVM. Rails and most other frameworks run. It's a Ruby for the java world and integrates well with java.
- 1.1 out a month ago. (Perf improvements.) The most compatible with Ruby 1.8. Future: more speed and comparability (groovy-level comparability). Share some low level advances with Groovy etc.
Jython: well structured, (honest). Wide range of libraries and applications. Can call into java.
- 2.2 is in prod. Behind phython. Large userbase. Trunk is 2.3 compatibility and aiming at 2.5. New compiler and parade being worked on. Already sharing a posix layer a. Jruby.
Rhino: dynamically typed and functional-like. Very flexible. Pervasive on the client, more on server coming up.
- 1.6r2 in Java 6. Interpreted by default but can be compiled. 1.7 will be java script 1.7 compatible. Ecmascript 2 is being tracked.
Case study:
JRuby. Dynamic typing is hard. Ruby has its own type system, continuations, etc. It also has Real world dependencies on c extensions.
But pushing boundaries on the jvm. This brings Java new blood, new ideas. Thus the JVM may be a better ruby runtime.
Complex: JRuvy does mixed mode- interpretataion, then compile later. Profiling is to be added. It has a custom string, array ( list), hash(map), regexp. It generates code to avoid reflection overhead. Multiple can run on a single JVM. Therefore you can have lots of Ruby apps on one JVM.
What would make it easier?:
- Lightweight code generation and loading.
- A faster-than-reflection method object.
- A tool chain for parsing interpretation and compiling.
- A more flexible JVM type system.
Collaboration: much is common across these languages. There are many similar problems. All are growing the platform. Consequently the platform is being pushed forward and extended.
Jvmlanguages group ( google group )
Jvm language runtime ( inspired by MSFT) should be general. We can gather the best libraries together and provide a fast track for implementation.
Possible new bits:
- Toolchain for parsing, interpretation and compiling: Antlr++?
- Language agnostic type inference: repurpose Scala or Haskell?
- Metaobject protocol? Flexible class definition. Learn from Groovy's MOP.
- Reflection-free method handles.
- Specialised types: c.f Ola's YAML parser
- The NLT is out there already
DaVinci VM: This is an open jdk based multi language vm.
It has anonymous classloading, lightweight method handles, optimised dynamic invocation, continuations, tail call optimisations, and tuples.
Your future: you are the community. You are not as dumb ss you think. You'll be a better person as a result. OSS is good for your career. You don't need to know about how to write a compiler. Be a polyglot - consider many languages for each problem.
Get involved: easy (core type systems) run rests and fix bugs. Interpreters are harder but basically the tree walker or walks the bytecode. Compilers are difficult but fun. Tree walk (easy) generate bytecode (hard and obscure)
What about IDE support?: generally driven by usage. Netbeans support is generic.
Language survey: 200+ lands
Nolanf is inpossible fix are most diff, dynamic are hard to optimise. No Lang is too weird
Functional Languages:
clojure:
Lisp dialect but easier to read and write.
- Released every 3-6 weeks. Getting exposure now.
Kawa: Implementation of scheme. One of oldest compilers.
- Currently v1.9.1. Multiple projects using it. Open secret.
Ocaml-java: Very expressive. Functional, imperative and OO styles, need to set up external files to bind to java calsses
- 1.0 out in beta now. 2.x branch comping to focus on speed.
Scala: OO, ( classes, traits and mixins)
Statically typed. Nice looking DSLs. Fully interoperable with Java
- 2.7.0 is final. 5000 downloads / month. The Lift web framework is gaining traction.
Groovy: many features from Python and Ruby. Always JVM Lang. Has closures, mutable classes with a very java-like syntax. Has many libraries built in.
- 1.5.6 is current. 1.6 is out in beta. (Perf improvements - 500> in some cases and tidied bytecode)
JRuby: Ruby on the JVM. Rails and most other frameworks run. It's a Ruby for the java world and integrates well with java.
- 1.1 out a month ago. (Perf improvements.) The most compatible with Ruby 1.8. Future: more speed and comparability (groovy-level comparability). Share some low level advances with Groovy etc.
Jython: well structured, (honest). Wide range of libraries and applications. Can call into java.
- 2.2 is in prod. Behind phython. Large userbase. Trunk is 2.3 compatibility and aiming at 2.5. New compiler and parade being worked on. Already sharing a posix layer a. Jruby.
Rhino: dynamically typed and functional-like. Very flexible. Pervasive on the client, more on server coming up.
- 1.6r2 in Java 6. Interpreted by default but can be compiled. 1.7 will be java script 1.7 compatible. Ecmascript 2 is being tracked.
Case study:
JRuby. Dynamic typing is hard. Ruby has its own type system, continuations, etc. It also has Real world dependencies on c extensions.
But pushing boundaries on the jvm. This brings Java new blood, new ideas. Thus the JVM may be a better ruby runtime.
Complex: JRuvy does mixed mode- interpretataion, then compile later. Profiling is to be added. It has a custom string, array ( list), hash(map), regexp. It generates code to avoid reflection overhead. Multiple can run on a single JVM. Therefore you can have lots of Ruby apps on one JVM.
What would make it easier?:
- Lightweight code generation and loading.
- A faster-than-reflection method object.
- A tool chain for parsing interpretation and compiling.
- A more flexible JVM type system.
Collaboration: much is common across these languages. There are many similar problems. All are growing the platform. Consequently the platform is being pushed forward and extended.
Jvmlanguages group ( google group )
Jvm language runtime ( inspired by MSFT) should be general. We can gather the best libraries together and provide a fast track for implementation.
Possible new bits:
- Toolchain for parsing, interpretation and compiling: Antlr++?
- Language agnostic type inference: repurpose Scala or Haskell?
- Metaobject protocol? Flexible class definition. Learn from Groovy's MOP.
- Reflection-free method handles.
- Specialised types: c.f Ola's YAML parser
- The NLT is out there already
DaVinci VM: This is an open jdk based multi language vm.
It has anonymous classloading, lightweight method handles, optimised dynamic invocation, continuations, tail call optimisations, and tuples.
Your future: you are the community. You are not as dumb ss you think. You'll be a better person as a result. OSS is good for your career. You don't need to know about how to write a compiler. Be a polyglot - consider many languages for each problem.
Get involved: easy (core type systems) run rests and fix bugs. Interpreters are harder but basically the tree walker or walks the bytecode. Compilers are difficult but fun. Tree walk (easy) generate bytecode (hard and obscure)
What about IDE support?: generally driven by usage. Netbeans support is generic.
JRuby at Thoughtworks - BOF-4807
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Ola Bini, Thoughtworks
Update: The slides are availble.
Alternative title: "Why Thoughtworks thinks (J)Ruby is worth investing in"
Thoughtworks - 40% of US Revenue comes from Ruby and Rails (more Ruby developers than any other company he knows of), deep involvement in Open Source, source of Selenium and Cruise Control
Product Development
- Mingle
- Cruise (re write of CC)
- Twist - new functional testing framework for Eclipse.
RubyWorks
- CruiseControl.rb (used on Rails project
- Ruby Production Stack (Like LAMP for Rails, plus takes care of security updates)
- JRuby - sponsors Ola's work on JRuby as a core developer
Why Ruby?
- RoR emphasises Agile practices
- Higher productivity
- drastically quicker time to market
- Extremely readable code (reading code is more important than writing it - read 100 times)
- State of the art testing libraries
- Well suited for prototyping
- Easy to implement DSLs (Martin Fowler is writing a book on it at the moment)
Problems With Ruby 0 Threading
- Ruby 1.8 - Green Threading
- Can't scale across processors / cores
- C libs can't / wont yield to ruby code
- 1.9: native, non parallel threads
- lots of work to ensure thread safety
Unicode Support:
- 1.8 Partial unicide
- Internet connected apps must have solid unicide
- App developers must roll their own: Rails Multibyte
- 1.9 Full unicode by drastic unicode
- String interface changes to per-char, not -per-byte
- Not backwards compatibile
Performance:
- 1.8 slower than most languages
- 1.9 improvement but still more to do
- New engine abiout 1.5 improvement
- inly implicit AOT ciompilation to bytecode
- more to do
Memory:
- 1.8 - simple design
- Good for many apps, but not scalable
- stop-the-world GC
- 1.9 no change
- Improved perf => more change
C-entenxsions:
- C is difficult to write well
- but there are many C extensions
- Beware! Badly behaved extensions can cause large problems
Politics:
- needs software, servers and training
- Legacy - lots of java apps in the world and extensive amount of Java frameworks
How JRuby solves them:
- Threading - Java threads are ruby threads. World class scheduler
- Unicide - JRuby uses Java Unicode
- Perf - AOT and JIT cimpoiler
- Memory - java provides world class GC
- Extensions - JRuby extensions are written in Java
- Credibility from running on JVM - knowing JRuby is there makes some Ruby projects OK to some customers
Typical Thoughtworks clients:
- Oracle (mix.oracle.com)
- LinkedIn
- Cisco
- Rackspace
Finaicial Institutions - banks, investment banks, caterpillar finaicial services
Other - SKF, Siemens
Thoughtworls Ruby Work:
- Web 2.0 apps
- internal prototypes
- small to medium web based apps / utilities
- "Soft layer APIs"
- Build systems for complex enterprise systems
Why JRuby?:
- Gives access to all java enterprise features
- conservative environments wqill not use MRO
- Quick integration witrh legacy systems
- Cost: Java+Ruby is more cost effective than MRI (using JRuby you get all good Ruby but with fallback to Java if you have a bottlekneck, legacy java libs, etc)
Case Study - Oracle mix:
- Social networking
- All oracle customers can connect
- Based on internal app (regular rails app) but now JRuby to run on ORacle stack (LDAP, SSL, DB, Linus etc)
- 4 person team (1 oracle) in 6 weeks
- JRuby on Rails
CS - Finance Client 1:
- IS standardised on java
- Glue app to improve integration
- Uses UnitRecord plugin to speed up test suites
- lack of good RMagik replacement has been hard (incomplete versions for JRuby)
Finance Client 2:
- Seperate business units
- aim to consolidate all accoutns, customer info in one place
- Ruby with small java
- 2 persons, 4 months (3 times linger in java)
- interacting with 5 disparate datastores
Why Ruby?
- "those infrastructure guys are just seeing a really large java app with a really large config file"
- Higher business va;lue for lower cost
- Using java apis (esp. JDBC) allowed quicker dev. since no need to write new functionality to Ruby
Case Study: Mingle:
- Agile in an easier way
- around almost a year
- chose JRuby due to lack of good SVN plugin
- Test suite (2.2 tests to code) slower in JRuby (due to lots of shallow func which test lots of pieces of an app. this pattern means the JIT compiler doesn't get any piurchase and most code is just interpreted (which is ~ 1.5 times slower))
- NOTE: Java 7 will move hotspot away frmo java - specific bytecode to more general constructs
- Licences use stong cryptography using a Java RSA library (would have been very hard in MRI but dead easy in JRuby)
- Override JRuby's LoadService which allows us to encrypt/decrypt all ruby files in app/*
- Will probably move to using AOT compilationwhen that is finished
- This is obfuscation - there is no real, safe protection in it
- ...but it seems to work well enough =)
- ChartDirector is a proprietary lib for maing charts
- Have a Java lib and a C extension lib for Ruby
0 A thins wrapper over the Java lib makes it possivle to use the same lib in JRuby and Ruby
- No MRI SVN lib which works on Mac, linux and wiondows
- Initial reason for gong with JRuby
- SVNKit is a java lib which provides SVN access on all java platforms
- install4j - installation and bindles JVM and jetty
- BYO database (used to be Derby - serious load testing broke it)
Mingle Demo:
- 2.0 released a few weeks ago
- Current release burn ups
- Card Wall - everything is a card which has properties
- Feature Trees
- Track bugs
- Where is the project right now
- Clients can also see it
- Because a physical card wall can;t scale outside a single room
Other Projects:
- Swiby (JRuby wrapper over Swing but with also a web and desktop interface)
- Waffle (Java web framework wihch uses JRuby to do controllers)
Challenges:
- Perf of Unit tests
- Solution t0 running MRI precommit ad Jruby in CI or using unitrecord instaed
- startip time (esp of rails) (Staged in background)
- Memory Consuption (high but still lower than mongrel)
- Good replacement for RMagik needed]
- Documentation (but a book is available - Practocal JRuby on Rails)
When to use Ruby?:
- interact wth Java
- you need to be really cross platform
- you are deploying into a Java-based organisation
When not use Ruby?:
- if you depend on unsupported C extenstions
- for smaller scripts
- if you need continuations
- corner cases on POSIX
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Ola Bini, Thoughtworks
Update: The slides are availble.
Alternative title: "Why Thoughtworks thinks (J)Ruby is worth investing in"
Thoughtworks - 40% of US Revenue comes from Ruby and Rails (more Ruby developers than any other company he knows of), deep involvement in Open Source, source of Selenium and Cruise Control
Product Development
- Mingle
- Cruise (re write of CC)
- Twist - new functional testing framework for Eclipse.
RubyWorks
- CruiseControl.rb (used on Rails project
- Ruby Production Stack (Like LAMP for Rails, plus takes care of security updates)
- JRuby - sponsors Ola's work on JRuby as a core developer
Why Ruby?
- RoR emphasises Agile practices
- Higher productivity
- drastically quicker time to market
- Extremely readable code (reading code is more important than writing it - read 100 times)
- State of the art testing libraries
- Well suited for prototyping
- Easy to implement DSLs (Martin Fowler is writing a book on it at the moment)
Problems With Ruby 0 Threading
- Ruby 1.8 - Green Threading
- Can't scale across processors / cores
- C libs can't / wont yield to ruby code
- 1.9: native, non parallel threads
- lots of work to ensure thread safety
Unicode Support:
- 1.8 Partial unicide
- Internet connected apps must have solid unicide
- App developers must roll their own: Rails Multibyte
- 1.9 Full unicode by drastic unicode
- String interface changes to per-char, not -per-byte
- Not backwards compatibile
Performance:
- 1.8 slower than most languages
- 1.9 improvement but still more to do
- New engine abiout 1.5 improvement
- inly implicit AOT ciompilation to bytecode
- more to do
Memory:
- 1.8 - simple design
- Good for many apps, but not scalable
- stop-the-world GC
- 1.9 no change
- Improved perf => more change
C-entenxsions:
- C is difficult to write well
- but there are many C extensions
- Beware! Badly behaved extensions can cause large problems
Politics:
- needs software, servers and training
- Legacy - lots of java apps in the world and extensive amount of Java frameworks
How JRuby solves them:
- Threading - Java threads are ruby threads. World class scheduler
- Unicide - JRuby uses Java Unicode
- Perf - AOT and JIT cimpoiler
- Memory - java provides world class GC
- Extensions - JRuby extensions are written in Java
- Credibility from running on JVM - knowing JRuby is there makes some Ruby projects OK to some customers
Typical Thoughtworks clients:
- Oracle (mix.oracle.com)
- Cisco
- Rackspace
Finaicial Institutions - banks, investment banks, caterpillar finaicial services
Other - SKF, Siemens
Thoughtworls Ruby Work:
- Web 2.0 apps
- internal prototypes
- small to medium web based apps / utilities
- "Soft layer APIs"
- Build systems for complex enterprise systems
Why JRuby?:
- Gives access to all java enterprise features
- conservative environments wqill not use MRO
- Quick integration witrh legacy systems
- Cost: Java+Ruby is more cost effective than MRI (using JRuby you get all good Ruby but with fallback to Java if you have a bottlekneck, legacy java libs, etc)
Case Study - Oracle mix:
- Social networking
- All oracle customers can connect
- Based on internal app (regular rails app) but now JRuby to run on ORacle stack (LDAP, SSL, DB, Linus etc)
- 4 person team (1 oracle) in 6 weeks
- JRuby on Rails
CS - Finance Client 1:
- IS standardised on java
- Glue app to improve integration
- Uses UnitRecord plugin to speed up test suites
- lack of good RMagik replacement has been hard (incomplete versions for JRuby)
Finance Client 2:
- Seperate business units
- aim to consolidate all accoutns, customer info in one place
- Ruby with small java
- 2 persons, 4 months (3 times linger in java)
- interacting with 5 disparate datastores
Why Ruby?
- "those infrastructure guys are just seeing a really large java app with a really large config file"
- Higher business va;lue for lower cost
- Using java apis (esp. JDBC) allowed quicker dev. since no need to write new functionality to Ruby
Case Study: Mingle:
- Agile in an easier way
- around almost a year
- chose JRuby due to lack of good SVN plugin
- Test suite (2.2 tests to code) slower in JRuby (due to lots of shallow func which test lots of pieces of an app. this pattern means the JIT compiler doesn't get any piurchase and most code is just interpreted (which is ~ 1.5 times slower))
- NOTE: Java 7 will move hotspot away frmo java - specific bytecode to more general constructs
- Licences use stong cryptography using a Java RSA library (would have been very hard in MRI but dead easy in JRuby)
- Override JRuby's LoadService which allows us to encrypt/decrypt all ruby files in app/*
- Will probably move to using AOT compilationwhen that is finished
- This is obfuscation - there is no real, safe protection in it
- ...but it seems to work well enough =)
- ChartDirector is a proprietary lib for maing charts
- Have a Java lib and a C extension lib for Ruby
0 A thins wrapper over the Java lib makes it possivle to use the same lib in JRuby and Ruby
- No MRI SVN lib which works on Mac, linux and wiondows
- Initial reason for gong with JRuby
- SVNKit is a java lib which provides SVN access on all java platforms
- install4j - installation and bindles JVM and jetty
- BYO database (used to be Derby - serious load testing broke it)
Mingle Demo:
- 2.0 released a few weeks ago
- Current release burn ups
- Card Wall - everything is a card which has properties
- Feature Trees
- Track bugs
- Where is the project right now
- Clients can also see it
- Because a physical card wall can;t scale outside a single room
Other Projects:
- Swiby (JRuby wrapper over Swing but with also a web and desktop interface)
- Waffle (Java web framework wihch uses JRuby to do controllers)
Challenges:
- Perf of Unit tests
- Solution t0 running MRI precommit ad Jruby in CI or using unitrecord instaed
- startip time (esp of rails) (Staged in background)
- Memory Consuption (high but still lower than mongrel)
- Good replacement for RMagik needed]
- Documentation (but a book is available - Practocal JRuby on Rails)
When to use Ruby?:
- interact wth Java
- you need to be really cross platform
- you are deploying into a Java-based organisation
When not use Ruby?:
- if you depend on unsupported C extenstions
- for smaller scripts
- if you need continuations
- corner cases on POSIX
Boldly go where the Java programming language has never gone before - TS-6213
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Geert Bevin, Terracota Inc.
AIM: Learn that you don't have to master new languages, tools and libraries to deliver applications that go much further than what the standard Java platform provides
NOTE: The Java Langauge != the JVM != the Java platform
- The langauge is for development
- The VM is for execution
- The platrofm is to transform what you've written into something which runs
- You can change the language - Groovy, JavaFXScript
- But you can also change the tooling - groovyc
- And you can also change the rutime - GCJ, Web Browser, Davlik
All of the layers can be replaced / extended
Today we'll focus on the java language as the constant
Terracotta:
- OSS clustering for Java Platform
- Scalability and availability for the JVM
- Terracotta cluster the JVM software (not wrap it via an external clustering app) and eliminate the need for custom code to access shared state
- Terracotta views each VM as a "thread" and creates a "superVM" across the network which allows different VM's to operate s if there were no machine boundaries at all
- Main features:
- Heap level replication (declarative, no serialisation, fine grained field level)
- Large virtual heaps (as large as available disk, dynamic paging)
- JVM co-ordination (distributed synchronised, distributed wait() / notify(), fine grained locking)
- Management
- [Example of shared state]
Terracotta's Boldness:
- plug into the JVM with a bootjar
- ...
RIFE Continuations:
- Full stack component framework to quickly and consistently develop and maintain java web applications
- What are continuations? - They are like "Save Game"
- They can be performed at any location, captures your posessions and progress, and you can have several at once
- But they are also like "load game"
- Retreive and saved game and resume where you were when you saved
- Game => Application
- Saved Game => Continuation
- Context and location are stored
- You can have multiple at once
- Used in RIFE to enable you to capture complex web application flows simply
- Continuations are only interesting if you start at an entry point (e.g. when a servlet executes rather than at the very beginning when Tomcat starts)
- You can use any java debugger and looping constructs in servlet code and it is all saved in your continuation without interrupting the web flow
- When you're done, remove continuations to stop abuse of the back button - destroys the "save gane"
RIFE's Boldness:
- Java method calls are trapped at runtime when classes are loaded to rewrite the code sections they're executing in
- Intuitive state handling, requiring no serialisation
- Regular java language statements are used for direct applciation control flow
- Allows you to use debuggers
- Provides continuation trees for backwards and forwards flow
GWT:
- Create AJAX apps - write in Java, compile to JS and run in any browser without running in a applet
- Features:
- Two main execution modes: Hosted (executed in the JVM, embedded browser hooks into java libraries) & Web (comiles and runs in browser without plugins)
- Standardised RPC - implement RemoteService interface, relies on serialisation for arguments and return values, supports asynchronous calls
- Code looks very much like Swing (component based plus listeners for event handling)
- Each java file has an associated HTML file which you place the components into. This HTML has div's with ID's which ties into the java placement code
- You can debug from your IDE when running hosted mode
- RPC: Asynchronous - Used for AJAX support
GWT's Boldness:
- Two execution modes to simplify dev and dep
- Browser is hooked into java libs to proide debugging during hsoetd mode
- The java code is compiled into java script
Google Android:
- A mobile platform for devices where sw is written in java but which runs on another VM with another type of bytecode
- Diff approach: OSS platform, all apps in Java programming language, dedicated VM per application (each clearly isolated)
- Very clearly defined concepts - ACTIVITY, VIEW, etc. - which your application must use
- Listener based (like GWT above)
- They ship a very nice emulator with the toolkit
- Again you can use your IDE's debugger with the emulator
Android's Boldness:
- They do it all in Java
- only use androids jRE which in cludes supported lib classes
- compile with javac and convert to android bytecode for dalvik vm (.dex files)
- debug with your debugger
Summary:
- using just Java and your IDE you can:
- cluster your app
- intuitive web apps with control flow which cross server / client barrier
- ajax client dev without javascript
- first class mobile app development
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Geert Bevin, Terracota Inc.
AIM: Learn that you don't have to master new languages, tools and libraries to deliver applications that go much further than what the standard Java platform provides
NOTE: The Java Langauge != the JVM != the Java platform
- The langauge is for development
- The VM is for execution
- The platrofm is to transform what you've written into something which runs
- You can change the language - Groovy, JavaFXScript
- But you can also change the tooling - groovyc
- And you can also change the rutime - GCJ, Web Browser, Davlik
All of the layers can be replaced / extended
Today we'll focus on the java language as the constant
Terracotta:
- OSS clustering for Java Platform
- Scalability and availability for the JVM
- Terracotta cluster the JVM software (not wrap it via an external clustering app) and eliminate the need for custom code to access shared state
- Terracotta views each VM as a "thread" and creates a "superVM" across the network which allows different VM's to operate s if there were no machine boundaries at all
- Main features:
- Heap level replication (declarative, no serialisation, fine grained field level)
- Large virtual heaps (as large as available disk, dynamic paging)
- JVM co-ordination (distributed synchronised, distributed wait() / notify(), fine grained locking)
- Management
- [Example of shared state]
Terracotta's Boldness:
- plug into the JVM with a bootjar
- ...
RIFE Continuations:
- Full stack component framework to quickly and consistently develop and maintain java web applications
- What are continuations? - They are like "Save Game"
- They can be performed at any location, captures your posessions and progress, and you can have several at once
- But they are also like "load game"
- Retreive and saved game and resume where you were when you saved
- Game => Application
- Saved Game => Continuation
- Context and location are stored
- You can have multiple at once
- Used in RIFE to enable you to capture complex web application flows simply
- Continuations are only interesting if you start at an entry point (e.g. when a servlet executes rather than at the very beginning when Tomcat starts)
- You can use any java debugger and looping constructs in servlet code and it is all saved in your continuation without interrupting the web flow
- When you're done, remove continuations to stop abuse of the back button - destroys the "save gane"
RIFE's Boldness:
- Java method calls are trapped at runtime when classes are loaded to rewrite the code sections they're executing in
- Intuitive state handling, requiring no serialisation
- Regular java language statements are used for direct applciation control flow
- Allows you to use debuggers
- Provides continuation trees for backwards and forwards flow
GWT:
- Create AJAX apps - write in Java, compile to JS and run in any browser without running in a applet
- Features:
- Two main execution modes: Hosted (executed in the JVM, embedded browser hooks into java libraries) & Web (comiles and runs in browser without plugins)
- Standardised RPC - implement RemoteService interface, relies on serialisation for arguments and return values, supports asynchronous calls
- Code looks very much like Swing (component based plus listeners for event handling)
- Each java file has an associated HTML file which you place the components into. This HTML has div's with ID's which ties into the java placement code
- You can debug from your IDE when running hosted mode
- RPC: Asynchronous - Used for AJAX support
GWT's Boldness:
- Two execution modes to simplify dev and dep
- Browser is hooked into java libs to proide debugging during hsoetd mode
- The java code is compiled into java script
Google Android:
- A mobile platform for devices where sw is written in java but which runs on another VM with another type of bytecode
- Diff approach: OSS platform, all apps in Java programming language, dedicated VM per application (each clearly isolated)
- Very clearly defined concepts - ACTIVITY, VIEW, etc. - which your application must use
- Listener based (like GWT above)
- They ship a very nice emulator with the toolkit
- Again you can use your IDE's debugger with the emulator
Android's Boldness:
- They do it all in Java
- only use androids jRE which in cludes supported lib classes
- compile with javac and convert to android bytecode for dalvik vm (.dex files)
- debug with your debugger
Summary:
- using just Java and your IDE you can:
- cluster your app
- intuitive web apps with control flow which cross server / client barrier
- ajax client dev without javascript
- first class mobile app development
Using Findbugs in Anger - TS-6590
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Bill Pugh, University of Maryland
AIM: Learn how to effectively use findbugs on a large project (100,000 + loc) and make effective use of the information it gives you
Static Analysis:
- It analyses your probgram without execution
- It doesn't need tests
- It doesn't (needen't) know what your program does
- It looks for violations of reasonable programming practices
Common (Incorrect) Wisdom about Bugs and Static Analysis
- Programmers are smart
- We have good techniquies to find bugs early
Why do Bugs Occur?
- Nobody is perfect
- THere are many types of common errors - misunderstood lang features, APIs, typos, misunderstood class or method invariants
- Everyne makes syntax errors but te compiler catches them - what about one step removed from a syntax error
Findbugs does scale:
- Sun, google and eBay use it on their entire java codebase
- google has fixed more than 1000 issues identified (and their code base is big)
Using Gindbugs Effectively:
- Static analysis isn't a silver bullet
- Other techniques are also valuable - tests, code review, etc
- Find the right combination - you have a finite and fixed budget and you want to find an effective / profitable way to use static analysis
- Make sure you get the most effedctive use out of this time
- Running analysis and finding stupid code is easy. But what is the code supposed to do? The hard part is often "who owns it?", "what is the code supposed to do?", "What is a test case to prove the bug?"
The Findbugs Ecosystem:
- is analyses classes (and jsp's if you compile them)so I can also analysis groovy, scala, jruby etc...
- filter files can be used to include / exclude certain issues
- output stored in XML files
- Many tools to post process the XML result
- WAys to perform analysis - Swing UI, CLI, Eclipse, Ant, Maven, NB, Cruise Control, Hudson
- CLI: using -xml:withMessages writes human readable message strings in the XML output. This is useful if any other tool other than findbugs will use the output
- Hudson: reads findbugs xml for each build and presents - warning trend graph, warning deltas for each graph, dashbaoard by package, link to source(working with Koshuke on integration with FB to make it even better)
Scaling up FB:
- Make it manageable. E.g. Eclipse 3.4M2 : 1. filter out low priority, 2. filter out vulnerability to malicious code (non final static variables), 3. filter out issues also present in Eclipse 3.3 (it shipped after QA so assume these ones are not that bad)
- Remember evaluations: if you want to evaluate an issue but don't immediately fix the code, you want to remember your evaluation - issues that must be addressed / fixed / reveiwed before the next release
- Highlight new issues - flag new as a result of latest commit (send email to comitter). Just keeping track of trend lines of total # of issues isn't good enough. If a change causes an issue, flag it. Hudson does this very well
- Integration with Bug Reporting / Tracking - scrape the XMl and import. Link the findbugs and bug db entry
- Typical FB warning density
- about 0.3 - 0.6 medium or high priority correctness warnings per 1000 lines of NCSS (non commenting source statements)
- About 1-4 other potentially relevant warnings per 1000 lines of code
- Don't use these numbers to judge whether your project is good or bad (lots of reasons results miht be biased)
- At Google
- Over 2 yesrs, perhaps 1 person year of effort on auditing issues
- Over that span reviewed 1663 issues - 804 fixed by developers
- What issues are you interested in?
- Priority - H/M/L. Looking at low pri not recc on large code bases. High/Med are useful for ranking within a pattern. Medium Foo issues might be more important than High Bar issues
- CAtegory - Correctness (developer probably made a mistake), Security (e.g. SQL injection & XSS), Bad Practice (code violates good practice - e.g. class with .equals() but no .hashCode()), Dodgy Code (code doing something unusual which may be incorrect - e.g. dead local store), Multithreaded Correctness (probs with synchronisation, notify() etc. but it is hard to do this with static analysis), Potential Perf. Probs (there is a more efficient way to do this), Malicious Code Vulnerability (static field which can be changed by untrusted code), Internationalisation
- Categories - Malicious code is v important if you run code in the same JVM as untrusted code. Perf issues are generally only importNT IN THE 10% OF YOUR CODE WHICH USED 90% OF YOUR cpu cycles (i.e. ignore static initialisation code)
- How to set up your config: Run first, then filter out the stuff you don't want
- Filtering can be simple or complex (e.g. in Ant or CLI). You can also use an xml filter file
- Filters can be "include" or "exclude" and can be used wen running the analysis, filtering bugs , and in the eclipse plugin
- Filter use cases - use to describe what is / isn't interesting. also use to filter out what has been reviewed and found not to be interesting
- The GUI can be used to build the filter files. Click on a bug and say "filter bugs like this" => select the attributes you want, and then add to the filter
- The GUI can also import and export filter files (to use in Eclipse, etc.)
Historical Bug Results:
- If you run FB as part of each build you can merge analysis results to combine into a bug history.
- You can then do queries on this history
- FB matches up corresponding bugs in successive versions - fuzzy match, not based on line numbers
- If a bug persists across multiple versions, the XML records the 1st and last version which contained the bug
- Querying: you can filter bugs basd on teh 1st and last version that contained an issue, or how it was introduced or removed
- Instance Hashes:
- when you generate an xml file with mesasges, an instance hash is associated with each bug. This is useful for connecting analysis results to bug databases and other forms of analysis processing
- Instance hash collisions - not guaranteed to be unique
- Unique identifiers. Each issue has an occurenceNum and an occurrenceMax as well as a hash. Concatenating all three gives something unique to the file and is unlikely to collide across successive versions
- Excluding baseline bugs:
- to look at e.g. just the bugs since relase 3.0 - too many to look at...
- estabish a bugs baseline and then "exclude all the issues in this bug file" (can be done in Eclipse) (based on the instance hash)
- Saving audit results:
- Sswing and Eclipse allow you to mark an issue: unclassified, bad analysis, should fix, must fix, etc...
- You can also add a free text annotation
- This is saved out in the XML and over history, these are matched up and combined
- The history is kept, but it doesn't provide an easy way to share it across workapsces. Relying on VCS to merge bug databases isn't recommended or supported. Plugin coming (this summer) to store this in an external database (plain text file, web server, etc.) to solve this
http://code.google.com/p/findbugs-tutorials
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Bill Pugh, University of Maryland
AIM: Learn how to effectively use findbugs on a large project (100,000 + loc) and make effective use of the information it gives you
Static Analysis:
- It analyses your probgram without execution
- It doesn't need tests
- It doesn't (needen't) know what your program does
- It looks for violations of reasonable programming practices
Common (Incorrect) Wisdom about Bugs and Static Analysis
- Programmers are smart
- We have good techniquies to find bugs early
Why do Bugs Occur?
- Nobody is perfect
- THere are many types of common errors - misunderstood lang features, APIs, typos, misunderstood class or method invariants
- Everyne makes syntax errors but te compiler catches them - what about one step removed from a syntax error
Findbugs does scale:
- Sun, google and eBay use it on their entire java codebase
- google has fixed more than 1000 issues identified (and their code base is big)
Using Gindbugs Effectively:
- Static analysis isn't a silver bullet
- Other techniques are also valuable - tests, code review, etc
- Find the right combination - you have a finite and fixed budget and you want to find an effective / profitable way to use static analysis
- Make sure you get the most effedctive use out of this time
- Running analysis and finding stupid code is easy. But what is the code supposed to do? The hard part is often "who owns it?", "what is the code supposed to do?", "What is a test case to prove the bug?"
The Findbugs Ecosystem:
- is analyses classes (and jsp's if you compile them)so I can also analysis groovy, scala, jruby etc...
- filter files can be used to include / exclude certain issues
- output stored in XML files
- Many tools to post process the XML result
- WAys to perform analysis - Swing UI, CLI, Eclipse, Ant, Maven, NB, Cruise Control, Hudson
- CLI: using -xml:withMessages writes human readable message strings in the XML output. This is useful if any other tool other than findbugs will use the output
- Hudson: reads findbugs xml for each build and presents - warning trend graph, warning deltas for each graph, dashbaoard by package, link to source(working with Koshuke on integration with FB to make it even better)
Scaling up FB:
- Make it manageable. E.g. Eclipse 3.4M2 : 1. filter out low priority, 2. filter out vulnerability to malicious code (non final static variables), 3. filter out issues also present in Eclipse 3.3 (it shipped after QA so assume these ones are not that bad)
- Remember evaluations: if you want to evaluate an issue but don't immediately fix the code, you want to remember your evaluation - issues that must be addressed / fixed / reveiwed before the next release
- Highlight new issues - flag new as a result of latest commit (send email to comitter). Just keeping track of trend lines of total # of issues isn't good enough. If a change causes an issue, flag it. Hudson does this very well
- Integration with Bug Reporting / Tracking - scrape the XMl and import. Link the findbugs and bug db entry
- Typical FB warning density
- about 0.3 - 0.6 medium or high priority correctness warnings per 1000 lines of NCSS (non commenting source statements)
- About 1-4 other potentially relevant warnings per 1000 lines of code
- Don't use these numbers to judge whether your project is good or bad (lots of reasons results miht be biased)
- At Google
- Over 2 yesrs, perhaps 1 person year of effort on auditing issues
- Over that span reviewed 1663 issues - 804 fixed by developers
- What issues are you interested in?
- Priority - H/M/L. Looking at low pri not recc on large code bases. High/Med are useful for ranking within a pattern. Medium Foo issues might be more important than High Bar issues
- CAtegory - Correctness (developer probably made a mistake), Security (e.g. SQL injection & XSS), Bad Practice (code violates good practice - e.g. class with .equals() but no .hashCode()), Dodgy Code (code doing something unusual which may be incorrect - e.g. dead local store), Multithreaded Correctness (probs with synchronisation, notify() etc. but it is hard to do this with static analysis), Potential Perf. Probs (there is a more efficient way to do this), Malicious Code Vulnerability (static field which can be changed by untrusted code), Internationalisation
- Categories - Malicious code is v important if you run code in the same JVM as untrusted code. Perf issues are generally only importNT IN THE 10% OF YOUR CODE WHICH USED 90% OF YOUR cpu cycles (i.e. ignore static initialisation code)
- How to set up your config: Run first, then filter out the stuff you don't want
- Filtering can be simple or complex (e.g. in Ant or CLI). You can also use an xml filter file
- Filters can be "include" or "exclude" and can be used wen running the analysis, filtering bugs , and in the eclipse plugin
- Filter use cases - use to describe what is / isn't interesting. also use to filter out what has been reviewed and found not to be interesting
- The GUI can be used to build the filter files. Click on a bug and say "filter bugs like this" => select the attributes you want, and then add to the filter
- The GUI can also import and export filter files (to use in Eclipse, etc.)
Historical Bug Results:
- If you run FB as part of each build you can merge analysis results to combine into a bug history.
- You can then do queries on this history
- FB matches up corresponding bugs in successive versions - fuzzy match, not based on line numbers
- If a bug persists across multiple versions, the XML records the 1st and last version which contained the bug
- Querying: you can filter bugs basd on teh 1st and last version that contained an issue, or how it was introduced or removed
- Instance Hashes:
- when you generate an xml file with mesasges, an instance hash is associated with each bug. This is useful for connecting analysis results to bug databases and other forms of analysis processing
- Instance hash collisions - not guaranteed to be unique
- Unique identifiers. Each issue has an occurenceNum and an occurrenceMax as well as a hash. Concatenating all three gives something unique to the file and is unlikely to collide across successive versions
- Excluding baseline bugs:
- to look at e.g. just the bugs since relase 3.0 - too many to look at...
- estabish a bugs baseline and then "exclude all the issues in this bug file" (can be done in Eclipse) (based on the instance hash)
- Saving audit results:
- Sswing and Eclipse allow you to mark an issue: unclassified, bad analysis, should fix, must fix, etc...
- You can also add a free text annotation
- This is saved out in the XML and over history, these are matched up and combined
- The history is kept, but it doesn't provide an easy way to share it across workapsces. Relying on VCS to merge bug databases isn't recommended or supported. Plugin coming (this summer) to store this in an external database (plain text file, web server, etc.) to solve this
http://code.google.com/p/findbugs-tutorials
Overview of JavaFX Script - TS-5152
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Christopher Oliver, Sun Microsystems
Philosphy:
F3 - express GUI's
Learn the lessons from natural language - make JavaFX Script expressive:
- There are rules but also irregularities
- There are function words
- Have both imperative (e.g. java) and declarative (express propositions about the world - functional) moods
... and from existing programming languages:
- Be concise - hit a mark somewhere between verbose (COBOL) and obscure (APL)
- Performance - Mediate between the needs of us and of the machine.
- Be imperative (the C family has won the familiar syntax war - play with the winner)
- But also borrow from declarative languages as well - both functional and niche languages (such as SQL, HTML, XPath/XQuery)
- Finally there is no pre processor so there can be no DSLs and no operator overloading. This means I can read the code no matter who wrote it. This avoids the "Tower of Babel" problem
The Compiler:
- It's OSS - http://openjfx-compiler.dev.java.net
- It's built on javac
- It has similar command line tools - javafxc, javafx, javafxdoc - These are analagous to Java CLI tools
FX Script Features:
- It is integrated with java (so all your java APIs are available to you)
- It is Object Oriented - this allows you to look at the world from the point of view of specific objects - from the 1st person perspective. Otherwise code is from a third person perspective
- It also provides closures & first class functions - borrowed from ECMA script
- There is also multiple inheritance - I may inherit from 1 java class & multiple Java interfaces OR multiple JavaFXScript classes (but no Java classes)
- It has both an imperative and declarative syntax
- It is statically typed (which will provide obvious IDE benefits)
- The concept of time is a first class type with memes provided to manage changes over time such as key frame animation (but without a dependency on graphics - this is used to manage state changes over time)
Declarative Syntax:
- It is an expression language - all constructs are expressions (like Ruby)
- Object literals - this is similar to javascript object literals but FXScript is statically typed. Therefore I can do things like: rectangle { x:10 y:10 }
- This allows me to specify an object graph / document tree declaratively and also in the proper place in the conceptual framework rather than a sequence of instructions out of line (as happens with Java)
- It also has a block expression - {} - delimited with sub expressions seperated by semi colons. It can contain any number of expressions within it. The last expression is the return value of the block. Blocks can be nested.
Collections:
- There are no arrays but FXScript has a construct called a sequence (which contains obejcts but sequences are not objects themselves). They are mutable. sequence comprehension (c.f. set comprehension) allows them to be generated from other sequences. The chosen syntax is like a 'for' loop
- The 'slice' function allows me to obtain a sub range of a sequence
- There are also range expression like in Ruby ([1..4])
- Also literals are declared in [] and seperated with ','
NOTE: Strings can add curly brace contained expressions which are evaluated into the string result (c.f. Ruby). You can do this to tag things to be internationalised / localised - no resource bundle / formatting pain...
Bindings:
- Now we have a declarative language to express a language graph (a scene is comprised of a set of GUI elements) how do we express dynamic elements? Bindings allow you to attach a dynamic expression to change values references in an expression and update object properties. Whenever something is modified, the bound values will be updated automatically
- Bound Function - can be used to create reusable code
- An unbound function is just a procedure like in Java - you can still use them in functions but variable changes dont trigger re evaluation
- A bound function - the return value of the return function, all variables in the closure will trigger it to be re evaluated. No void expressions, no loops, etc.
Example:
- We have a conceptual line, with bound functions which extrapolate values from the features of the line (e.g. length, centrepoint). Some of these depend on other bound functions
- We firstly create the line, and then create the visual properties which depend upon the line. If we change one of the attributes of the line the visual elements will be automatically be updated
- NOTE: a group is a non visual scene graph node which creates a co ordinate space for the elements contained
Imperative Syntax:
- This is like java - if statements, while loops etc.
- There are also unbound functions
- And void expressions - which allow you to express procedural code
- NOTE: JavaFX Script objects are mutable, NOT immutable
- What about sequence modification? - the 'insert' and 'delete' keywords borrowed from SQL for insertion and deletion. Lesson from natural languages
- There are also triggers - Remember your program is about handling cause and effect; your program must handle change. There are already declarative bindings to link one object to another (like the Observer pattern). We can also respond in a procedural way, so if something happens (a trigger) 'on replace' then fires off some procedural function. The body of triggers are void block expressions. There was a change and this is executed to make a further change based on this triggering change
Key Frame Animation:
- This is another mechanism to manage state changes.
- Just to do with object state.
- element: timeline, element: keyframe
- set of states represented as variables 9at a certain time the variable had a certain value. when I run the timeline, at the instance of the keyframes the variables will be assigned the specified values.
Time literatls ('s' is seconds, 'ms' is millis)
[Keyframe {
values: [angle => 0, ...
],
[Keyframe {
values: [angle => 2, ...
- The tween operator - will animate the gaps in the timeline
Questions:
- Do you support generics? No, there is no support yet in JavaFX Script
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Christopher Oliver, Sun Microsystems
Philosphy:
F3 - express GUI's
Learn the lessons from natural language - make JavaFX Script expressive:
- There are rules but also irregularities
- There are function words
- Have both imperative (e.g. java) and declarative (express propositions about the world - functional) moods
... and from existing programming languages:
- Be concise - hit a mark somewhere between verbose (COBOL) and obscure (APL)
- Performance - Mediate between the needs of us and of the machine.
- Be imperative (the C family has won the familiar syntax war - play with the winner)
- But also borrow from declarative languages as well - both functional and niche languages (such as SQL, HTML, XPath/XQuery)
- Finally there is no pre processor so there can be no DSLs and no operator overloading. This means I can read the code no matter who wrote it. This avoids the "Tower of Babel" problem
The Compiler:
- It's OSS - http://openjfx-compiler.dev.java.net
- It's built on javac
- It has similar command line tools - javafxc, javafx, javafxdoc - These are analagous to Java CLI tools
FX Script Features:
- It is integrated with java (so all your java APIs are available to you)
- It is Object Oriented - this allows you to look at the world from the point of view of specific objects - from the 1st person perspective. Otherwise code is from a third person perspective
- It also provides closures & first class functions - borrowed from ECMA script
- There is also multiple inheritance - I may inherit from 1 java class & multiple Java interfaces OR multiple JavaFXScript classes (but no Java classes)
- It has both an imperative and declarative syntax
- It is statically typed (which will provide obvious IDE benefits)
- The concept of time is a first class type with memes provided to manage changes over time such as key frame animation (but without a dependency on graphics - this is used to manage state changes over time)
Declarative Syntax:
- It is an expression language - all constructs are expressions (like Ruby)
- Object literals - this is similar to javascript object literals but FXScript is statically typed. Therefore I can do things like: rectangle { x:10 y:10 }
- This allows me to specify an object graph / document tree declaratively and also in the proper place in the conceptual framework rather than a sequence of instructions out of line (as happens with Java)
- It also has a block expression - {} - delimited with sub expressions seperated by semi colons. It can contain any number of expressions within it. The last expression is the return value of the block. Blocks can be nested.
Collections:
- There are no arrays but FXScript has a construct called a sequence (which contains obejcts but sequences are not objects themselves). They are mutable. sequence comprehension (c.f. set comprehension) allows them to be generated from other sequences. The chosen syntax is like a 'for' loop
- The 'slice' function allows me to obtain a sub range of a sequence
- There are also range expression like in Ruby ([1..4])
- Also literals are declared in [] and seperated with ','
NOTE: Strings can add curly brace contained expressions which are evaluated into the string result (c.f. Ruby). You can do this to tag things to be internationalised / localised - no resource bundle / formatting pain...
Bindings:
- Now we have a declarative language to express a language graph (a scene is comprised of a set of GUI elements) how do we express dynamic elements? Bindings allow you to attach a dynamic expression to change values references in an expression and update object properties. Whenever something is modified, the bound values will be updated automatically
- Bound Function - can be used to create reusable code
- An unbound function is just a procedure like in Java - you can still use them in functions but variable changes dont trigger re evaluation
- A bound function - the return value of the return function, all variables in the closure will trigger it to be re evaluated. No void expressions, no loops, etc.
Example:
- We have a conceptual line, with bound functions which extrapolate values from the features of the line (e.g. length, centrepoint). Some of these depend on other bound functions
- We firstly create the line, and then create the visual properties which depend upon the line. If we change one of the attributes of the line the visual elements will be automatically be updated
- NOTE: a group is a non visual scene graph node which creates a co ordinate space for the elements contained
Imperative Syntax:
- This is like java - if statements, while loops etc.
- There are also unbound functions
- And void expressions - which allow you to express procedural code
- NOTE: JavaFX Script objects are mutable, NOT immutable
- What about sequence modification? - the 'insert' and 'delete' keywords borrowed from SQL for insertion and deletion. Lesson from natural languages
- There are also triggers - Remember your program is about handling cause and effect; your program must handle change. There are already declarative bindings to link one object to another (like the Observer pattern). We can also respond in a procedural way, so if something happens (a trigger) 'on replace' then fires off some procedural function. The body of triggers are void block expressions. There was a change and this is executed to make a further change based on this triggering change
Key Frame Animation:
- This is another mechanism to manage state changes.
- Just to do with object state.
- element: timeline, element: keyframe
- set of states represented as variables 9at a certain time the variable had a certain value. when I run the timeline, at the instance of the keyframes the variables will be assigned the specified values.
Time literatls ('s' is seconds, 'ms' is millis)
[Keyframe {
values: [angle => 0, ...
],
[Keyframe {
values: [angle => 2, ...
- The tween operator - will animate the gaps in the timeline
Questions:
- Do you support generics? No, there is no support yet in JavaFX Script
Mylyn: Code at the Speed of Thought - TS-6421
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Mik Kerste, Tasktop Technologies
Key Concept: Flow + Focus = :-)
It would be nice if the tools could helop you get into and maintain this state
Friction - whenever the environment stops the flow. Machine slowness, administration (configuring), constant seearching
Distraction - affects focus (IM, email, bug reports, etc.). Multitasking, Interruptions, Email/IM overload
Reduce Friction - faster CPU, Java and tool support means tools automate more plus OS's are improving
My Brain's Long Term Memory:
- Semantic Memory - understanding of facts. How we understand the JDK and libraries. Takes reinforcement to learn
- IDEs make the semantic structure of our systems explicit to help us
- But the ystems we build are not getting smaller and the tools don't filter in their tree views
- The tree views are getting more and more full. We spend more time browsing and navigating
- Episodic Memory - our personal memory. Mainly hippocampal. One shot learning memory. We don't have to try and remember it. You can lose it in trauma cases.
- How can we leverage this in IDEs? For every episode that we work on (feature, bug, ...) we only care about the subset of the system related to that episode.
- Episodes are called "tasks" and you activate them with a click. Thsi is a subtle change in your workflow. Thence all the tree views in the rest of the IDE just show what is relevant to you ("Degree of interest model")
Demo:
=> Task focussed interface - Task List View. Task Respositories (inc. gmail, SVN, CVS, Bugzilla, JIRA)
- Aim to make everything sub 200ms reaction time - cached locally and synched offline
- Concept of incoming and outgoing changes
- All built on eclipse SWT and Workbench
- You can set up searches to find tasks (i.e. all open for release 3.0)
- Concept of working sets - so you can have a single task list but with different perspective
Task Context:
- Select and activate - Package explorer now just shows all the code I was working on when I last worked on the task. The task context.
- When I navigate around the editor, the relevant pieces in the navigator update and expand
- To see everything you can ALT-click something (but this is usually hidden)
- The context is autoamtically stored
- The degree of interest model decays. The more interesting something is, the more it decats
- When I deactivate and then reactivate a task, it remembers exactly where I was, right down to the eclipse extensions, what code I had selected, etc.
Collaboration:
- Synchronise view now grouped by task so I can commit just one task. Mylyn can write the commit message for you. (You can edit these in the preferences window)
- I can make a patch for the changeset in a right click
- Now changesets are units
- SVN history can be linked to the code and so now the change history is explicit in your code and easy to navigate to / from it
- Task contexts are sharable
Eclipse Europa:
- Now bundled in Eclipse Europa
- It is based on a framework:
- It needs to support tasks and the resources that you use
- Bugzilla and Java Reference Implementations - functionality bootstrapped from here
- Community provided JIRA connector
- Trac from Google code
- Projects outside Mylyn built on the APIs - Gmail, Gcal, browser, FogBugz, Outlook, Subclipse, Subversive, SpringIDE, Mantis, CodeBeamer, RT and more
- org.eclipse.mylyn
- core, rcp, sdk - Standard Eclipse platform
- monitor.core (monitors user ineration)
- cnoetxt.core (degree of interest etc.) and tasks.core (tasks and offline access)
- monitor.ui, context.ui, tasks.ui, and to integrate for example with Jira - jira.ui
- (extend into the IDE) monitor.usage, resources.ui, team.ui, team.cvs, ide.ui, java.ui
Summary:
- Tasks become a first class part of the UI
NOTE: Rally - brings Agile terminology into the Task list (user stories etc.)
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Mik Kerste, Tasktop Technologies
Key Concept: Flow + Focus = :-)
It would be nice if the tools could helop you get into and maintain this state
Friction - whenever the environment stops the flow. Machine slowness, administration (configuring), constant seearching
Distraction - affects focus (IM, email, bug reports, etc.). Multitasking, Interruptions, Email/IM overload
Reduce Friction - faster CPU, Java and tool support means tools automate more plus OS's are improving
My Brain's Long Term Memory:
- Semantic Memory - understanding of facts. How we understand the JDK and libraries. Takes reinforcement to learn
- IDEs make the semantic structure of our systems explicit to help us
- But the ystems we build are not getting smaller and the tools don't filter in their tree views
- The tree views are getting more and more full. We spend more time browsing and navigating
- Episodic Memory - our personal memory. Mainly hippocampal. One shot learning memory. We don't have to try and remember it. You can lose it in trauma cases.
- How can we leverage this in IDEs? For every episode that we work on (feature, bug, ...) we only care about the subset of the system related to that episode.
- Episodes are called "tasks" and you activate them with a click. Thsi is a subtle change in your workflow. Thence all the tree views in the rest of the IDE just show what is relevant to you ("Degree of interest model")
Demo:
=> Task focussed interface - Task List View. Task Respositories (inc. gmail, SVN, CVS, Bugzilla, JIRA)
- Aim to make everything sub 200ms reaction time - cached locally and synched offline
- Concept of incoming and outgoing changes
- All built on eclipse SWT and Workbench
- You can set up searches to find tasks (i.e. all open for release 3.0)
- Concept of working sets - so you can have a single task list but with different perspective
Task Context:
- Select and activate - Package explorer now just shows all the code I was working on when I last worked on the task. The task context.
- When I navigate around the editor, the relevant pieces in the navigator update and expand
- To see everything you can ALT-click something (but this is usually hidden)
- The context is autoamtically stored
- The degree of interest model decays. The more interesting something is, the more it decats
- When I deactivate and then reactivate a task, it remembers exactly where I was, right down to the eclipse extensions, what code I had selected, etc.
Collaboration:
- Synchronise view now grouped by task so I can commit just one task. Mylyn can write the commit message for you. (You can edit these in the preferences window)
- I can make a patch for the changeset in a right click
- Now changesets are units
- SVN history can be linked to the code and so now the change history is explicit in your code and easy to navigate to / from it
- Task contexts are sharable
Eclipse Europa:
- Now bundled in Eclipse Europa
- It is based on a framework:
- It needs to support tasks and the resources that you use
- Bugzilla and Java Reference Implementations - functionality bootstrapped from here
- Community provided JIRA connector
- Trac from Google code
- Projects outside Mylyn built on the APIs - Gmail, Gcal, browser, FogBugz, Outlook, Subclipse, Subversive, SpringIDE, Mantis, CodeBeamer, RT and more
- org.eclipse.mylyn
- core, rcp, sdk - Standard Eclipse platform
- monitor.core (monitors user ineration)
- cnoetxt.core (degree of interest etc.) and tasks.core (tasks and offline access)
- monitor.ui, context.ui, tasks.ui, and to integrate for example with Jira - jira.ui
- (extend into the IDE) monitor.usage, resources.ui, team.ui, team.cvs, ide.ui, java.ui
Summary:
- Tasks become a first class part of the UI
NOTE: Rally - brings Agile terminology into the Task list (user stories etc.)
Pimp My Build - TS-5596
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Matt Quail & Conor MacNeil, Atlassian
You might Learn
- Ways to improve your Ant build
- How to make your Maven build work
#1 - Use Imports and Macros
#3 - Don't build stuff you don't need
#4 - Spice up your build
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Matt Quail & Conor MacNeil, Atlassian
You might Learn
- Ways to improve your Ant build
- How to make your Maven build work
#1 - Use Imports and Macros
- to import the files containing macros you are using in your build
- means other folks can reuse (on diff projects even) your macros without knowing about their details
- and keeps your main ant.xml clean and simple and factors out commonality
- Tip!: add some guards in the macro to make sure stuff won't fail
- You get:
- A modularised build (kind of like Maven)
- Move all the targets that don't change across your builds into an importable module
- Import the module and override as appropriate
#3 - Don't build stuff you don't need
- use ant's "uptodate" task
- it is complex
- invest the time to get it right in the build
- keep you and your team productive
- the build will remember how to do it
- seperate it out into a macro and make its use simple
- also use outofdate from ant-contrib
- even better
#4 - Spice up your build
- Show a splash screen while the build is running
- nice and cute
- iIf a test fails - play "blame train" audio
- Add a splash image
- and associate it with a sprint, scrum iteration, etc.
- Can I do this with maven?:
- Yup, use growler (run the output of maven through an awk script which pattern matches and push out to growlnotify script)
- Can I use growl with ant?:
- yup
- Don't seperate the team into builders and coders
- everyone should know how the build works
- the build should not be precious or off limits
- Ant lests you define tasks with scripts
- e.g. in javascript (in BSF in ANT)
- Yes, you should try to be declarative but...
- e.g. you are dependent upon the OS platform => use the "if" task
- use the ant-contrib tasks (where "if" is from)
- Your build is a way to record how to do a set of complex operations
- There are heaps of analysis tools out there - USE THEM
- Put them in your build, in your CI, generate reports
- Ant allows you to put descriptions on all your targets
- Add comments
- Use the -target_name convention for private targets
- can't be invoked from the command line
- no its not perfect
- yes it does work
- its called an idiom
- Remember your build is where you document and remember things
- run tests
- run the build
- create the licence file
- generate coverage reports
- attach debuggers, run profilers
- Gives newbies a leg up
- Continuous Integration
- Controlled build environent
- Automated testing and reporting - its easy because its already in your build file
- Use a remote repository proxy (e.g. apache archiva) for stability and performance
- Create a local repository for private artefacts
- your own artefacts
- missing third party artitafcts
- Create a local repository for public artefacts
- artifacts you are making available
- You need to manage your build infrastructure
Improving the Engineering Process through Automation by Hudson - TS-6547
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
GOAL: Do more, with less
What is Hudson?
- Computers are getting cheaper but people are getting more expensive. We need to find ways to put computers to work for us
- Hence continuous Integration. Nightly builds? Dont be so cheap! We want software to do this
- Hudson is a java based CI server - looks for changes in VCS, runs tests and aggregates results, records all the data on one web site
- Hudson reduces communication overhead - how to build, how to test. Reduce the people dependency by removing tacit knowledge, especially for distributed teams
- Hudson is like another developer on your team (or your butler ... hence the name)
- Hudson is an OSS project on Java.net
- Hudson places particular emphasis on ease of installation and use
- Extensible architecture - 50+ community developed plugins
- Very active - 150 person years worth of code, FWIW, 3 years old, > 210 releases to date, 50+ committers
- Localised to 8 languages
- Helping all kinds of businesses - amazon, yahoo, NB, glassfish, mysql, linkedin, jboss
- Basic feature set:
- check out source code: cvs, SVN, clearcase, mercurial, etc.
- Do builds: ant, maven, shell script, Nant, ...
- record results
- close the feedback loop: RSS feeds, EMail/IM notification, Tray app to notify developers
- Tool integration: findbugs, emma, cobertura, VMWare
Demo
- hudson.dev.java.net
- download the hudson.war
- invoke java -jar hudson.war and you're running!
- or you can deploy it to your web container
- then go to localhost:8080
- Front page: see status of all jobs
- RESTful and intuitive URLS: e.g. invoke http://localhost:8080/job/your_project/build URL to start a build automatically, .../configure is the configuration page
- Test fails have "age" attribute - you can see how many builds ago the failure first occurred
- Works nicely with maven 2 projects - it can parse the POMs and figure out what you are trying to do and automatically do the reporting for you
Best Practices
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
GOAL: Do more, with less
What is Hudson?
- Computers are getting cheaper but people are getting more expensive. We need to find ways to put computers to work for us
- Hence continuous Integration. Nightly builds? Dont be so cheap! We want software to do this
- Hudson is a java based CI server - looks for changes in VCS, runs tests and aggregates results, records all the data on one web site
- Hudson reduces communication overhead - how to build, how to test. Reduce the people dependency by removing tacit knowledge, especially for distributed teams
- Hudson is like another developer on your team (or your butler ... hence the name)
- Hudson is an OSS project on Java.net
- Hudson places particular emphasis on ease of installation and use
- Extensible architecture - 50+ community developed plugins
- Very active - 150 person years worth of code, FWIW, 3 years old, > 210 releases to date, 50+ committers
- Localised to 8 languages
- Helping all kinds of businesses - amazon, yahoo, NB, glassfish, mysql, linkedin, jboss
- Basic feature set:
- check out source code: cvs, SVN, clearcase, mercurial, etc.
- Do builds: ant, maven, shell script, Nant, ...
- record results
- close the feedback loop: RSS feeds, EMail/IM notification, Tray app to notify developers
- Tool integration: findbugs, emma, cobertura, VMWare
Demo
- hudson.dev.java.net
- download the hudson.war
- invoke java -jar hudson.war and you're running!
- or you can deploy it to your web container
- then go to localhost:8080
- Front page: see status of all jobs
- RESTful and intuitive URLS: e.g. invoke http://localhost:8080/job/your_project/build URL to start a build automatically, .../configure is the configuration page
- Test fails have "age" attribute - you can see how many builds ago the failure first occurred
- Works nicely with maven 2 projects - it can parse the POMs and figure out what you are trying to do and automatically do the reporting for you
Best Practices
- Dependency tracking - have build jobs and seperate test jobs. If test jobs take longer than builds, it is important to know what build is being tested by a job. Hudson can tell you using MD5 checksumming so you can tell what build is being tested
- Baseline - newbies do everything inside a single job (e.g. scm update, ant/maven, test, findbugs, done). => Compartmentalisation:
- full builds (before you split into pieces) (e.g. Metro)
- alongside this, have sub builds (e.g. JAXB and JAX-WS) running
- this gives faster feedback (the point of CI) and isolates your problems nicely so the rest of a team can forge ahead while the right person fixes the build
- Test / Build separation:
- tests often dominate the build time
- start test jobs once the build completes
- then you can run them in parallel
- what if there are many alternatives? e.g.: unit test with GF, unit test with tomcat, SQE test, test w/o container (3 OS * 2 JDK = 6 tests)
- Enable fingerprinting of the artifact bits: so that hudson knows which test run is testing which build
- Copy bits from the build to the test dir with wget
- "Does this build work?" - just becasue the build worked does not mean it was "good"
- QA takes time but CI should be fast
- Old way - set up "nighlies" but this is disconnected from CI builds (diff build #s etc.) and duplicates the build setup
- New Way - Build promotion
- Allied army at Omaha Bean model - throw everything at it and see how far each of the individual parts get
- Produce the builds as quickly as you can. If a build passes tests, promote it. If it fails, it is "killed"Once promoted all promoted builds are listed on the website
- then commence more expensive QA, deploy to staging server, integrate, push to maven repository, etc.
- Distributed Builds:
- Master (servers HTTP requests, stores all critical info like config) and slaves (83K single jar deployment, co ordinates local processes, can come and go easliy)
- If there are more jobs than hudson can execute in parallel, hook up more slaves
- slaves launch via java web start
- MultiConfigJobs allow Hudson to take advantages of slaves to build s/w against different environments (e.g. Operating Systems, databases, etc.)
- you set up your multiple configurations build matrix in the build config page - you just tick the build parameters you want.
- the results are reported in a single page as a traffic-light matrix, against all the different environments
- Navigation - use the search panel in hudson or in your browser search bar
- remoting API
- RESTful API to retreive / push data as XMl or JSON
- Eclipse and NB plugins are available build on this API
- Integrate with issue trackers - link to Jira and have Hudson update the Jira issues
- maven plugin for - initial skeleton creation, handle compile time annotaton processing, debug support, package, release
The Future of Guice - BOF-6400
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
"Crazy" Bob Lee, Kevin Bourrillion andJesse Wilson (of Glazed Lists fame)
Guice 1.0
- it was v. popular
- it recently won a jolt award
- there are two books already available
Philosophy
- back to basics
- @inject is the new "new" - it has the brevity of "new" but the flexibility of a factory
- Fail early but not too early (validate all your dependencies at startup.)
- Make it easy to do the right thing
- Types are the natural currency of Java - do everything with types (early versions were String based) which means that your existing IDE (without plugins) still work
- Prefer annotations to convention - i.e. it's not just a naming convention
- Singletons aren't bad, but the typical implementations are
- Focus on readability over writeability
- Maximise the power-to-weight ratio - the Guice API is very small. If you'll never use it, the API does not expose it. Thus it is simple to maintain compatibility between versions
- Balance - just because you can doesn't mean you should
Goals for 2.0
- Releive pain points - implementing custom factories (maintainability)
- Compatability with 1.0 -but we may break where you want us to, e.g. they forgot about inner nested classes
- Extensibility - 1.0 made it easy to build a "Guicy" app. Now it's easier to extend it
- Tool support - There was an Eclipse plugin developed last year by a Google intern. They will provide all the hooks a tool needs to have
- Shooting for a summer 2008 release
Guice 2.0 Core
- Mirror API - SPI for Guice. If you want deep integrations with Guice to extend it or to hook it into a third party tool. Deep access to what's going on in Guice
- Tool stage - a mechanism to create injectors for tools to use. Possible since Guice uses Java code to specify its injection but in a shallow way - it won't start your singletons etc.
- Binder.getProvider()- if your module needs a binding to another module. This is an API so your module can get at other dependencies in other modules
- Class/Constructor Listeners - if you build an arbitrarily complex system (EJB 3.0) you can use Guice as a hosting environment. You will be able to do all the cool APOO reflection without all the weight. Powerful way to write aspect code. Its not APO, but it allows you to get a handle on a newly created object befiore its returned from construction
- Multibindings - lets you bind the same type n times which allows Guice to do another job in your system
- Constant Type Converters - you can already take a file of properties; but unless your properties are booleans or integers etc. you inject Strings. now you can register a converter for e.g. HTML converters and inject actual colour objects from a properties file.
- @Nullable - Now you can inject null
- Enhanced Error Reporting - it was really really good. It's now even better
- Specified exception reporting - whatever you throw from provider.get() method is a ProviderException.
- Module Overrides - Someone else gives you a module which has a dependency which you don't want. Whenever it binds to this type, you can swap out the pieces you don't want
Provider Methods
- These are all tidied up now. Now they're all beautiful and terse
Hierarchical Injectors
- A building block for power features
- Private bindings
- capability based security a la
- Just think robot legs
- Struts 2 plugin
- Business Logic vs. View Layer - one injector for each, ensuring the injectors for one area don't pollute the namespace for the other
Beyond 2.0
- Compile-time Guice - if you hate reflection (i.e. Mobile and embedded ot GWT) then you can take a Guice module and compile it to code which you can use
- Third party extensions
- Warp
- Guiceberry - Useing Guice to inject unit, functional, and integration tests
- Peaberry OSGi
- Platform support
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
"Crazy" Bob Lee, Kevin Bourrillion andJesse Wilson (of Glazed Lists fame)
Guice 1.0
- it was v. popular
- it recently won a jolt award
- there are two books already available
Philosophy
- back to basics
- @inject is the new "new" - it has the brevity of "new" but the flexibility of a factory
- Fail early but not too early (validate all your dependencies at startup.)
- Make it easy to do the right thing
- Types are the natural currency of Java - do everything with types (early versions were String based) which means that your existing IDE (without plugins) still work
- Prefer annotations to convention - i.e. it's not just a naming convention
- Singletons aren't bad, but the typical implementations are
- Focus on readability over writeability
- Maximise the power-to-weight ratio - the Guice API is very small. If you'll never use it, the API does not expose it. Thus it is simple to maintain compatibility between versions
- Balance - just because you can doesn't mean you should
Goals for 2.0
- Releive pain points - implementing custom factories (maintainability)
- Compatability with 1.0 -but we may break where you want us to, e.g. they forgot about inner nested classes
- Extensibility - 1.0 made it easy to build a "Guicy" app. Now it's easier to extend it
- Tool support - There was an Eclipse plugin developed last year by a Google intern. They will provide all the hooks a tool needs to have
- Shooting for a summer 2008 release
Guice 2.0 Core
- Mirror API - SPI for Guice. If you want deep integrations with Guice to extend it or to hook it into a third party tool. Deep access to what's going on in Guice
- Tool stage - a mechanism to create injectors for tools to use. Possible since Guice uses Java code to specify its injection but in a shallow way - it won't start your singletons etc.
- Binder.getProvider()- if your module needs a binding to another module. This is an API so your module can get at other dependencies in other modules
- Class/Constructor Listeners - if you build an arbitrarily complex system (EJB 3.0) you can use Guice as a hosting environment. You will be able to do all the cool APOO reflection without all the weight. Powerful way to write aspect code. Its not APO, but it allows you to get a handle on a newly created object befiore its returned from construction
- Multibindings - lets you bind the same type n times which allows Guice to do another job in your system
- Constant Type Converters - you can already take a file of properties; but unless your properties are booleans or integers etc. you inject Strings. now you can register a converter for e.g. HTML converters and inject actual colour objects from a properties file.
- @Nullable - Now you can inject null
- Enhanced Error Reporting - it was really really good. It's now even better
- Specified exception reporting - whatever you throw from provider.get() method is a ProviderException.
- Module Overrides - Someone else gives you a module which has a dependency which you don't want. Whenever it binds to this type, you can swap out the pieces you don't want
Provider Methods
- These are all tidied up now. Now they're all beautiful and terse
Hierarchical Injectors
- A building block for power features
- Private bindings
- capability based security a la
- Just think robot legs
- Struts 2 plugin
- Business Logic vs. View Layer - one injector for each, ensuring the injectors for one area don't pollute the namespace for the other
Beyond 2.0
- Compile-time Guice - if you hate reflection (i.e. Mobile and embedded ot GWT) then you can take a Guice module and compile it to code which you can use
- Third party extensions
- Warp
- Guiceberry - Useing Guice to inject unit, functional, and integration tests
- Peaberry OSGi
- Platform support
Thursday, May 08, 2008
Applets Reloaded - TS-6290
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
- Ken Russell & Sven Goethel
Intro
- Applets in Java6_u10 is a ground up re-write
- Applets no longer ru the browser process. (in the classic java browser plugin all the applets ran in same VM in the browser process)
- Now there is a thin JVM broker in the browser process and you can launch a new (seperate) VM per applet when required
- Advantages: Improved reliability (no browser crashes), no browser hang when loading, built in JNLP support (specify command line args on per applet instance), improved java/javascripts bridge and improved Vista support (signed applets run in protected mode IE)
Deploying via JNLP
- You can launch applets directly from JNLP files
- These follow the same descriptor as those in the Java Web Start architecture (therefore you can run your app as an applet or JWS application)
- Referencing resources - You can also reference your resources using relative (rather than absolute as with the old plugin) code paths. This allows simple migration of content from server to server and place to place
- There is also a new applet-desc tag. This references the main class, width, height
- There is also an information tag and a resources tag which references the other jar files providing info info like eager loading etc.
Command Line Arguments
- You can now define the CL args used to start the applet's JVM such as heap size, GC type, JRE version, plus properties - e.g. if you want to start an OpenGL applet
- The plugin launches a new JVM machine instance when necessary (separate_jvm tag)
- Demo: Quake 2 now runs in an applet - this shows the isolation and seperate JVM features
- The end user can specify CLIs too. There is a secure and a non-secure set.
- You must sign your jar and jnlp descriptor to get the capability to get an unrestricted JVM for your app
JRE Version Selection
- This allows you to run a particular applet in a particular version of the JRE
- This is useful for enterprises who want to QA applets against a particular JRE version
- It uses 'plus' semantics so you can have a JRE or later, or a specific family (1.4, 1.5 etc.)
- Auto download of requried but not installed JREs is supported by JNLP - you can also reference an installer for JWS
- Versions?: As early as 1.4.2 is supported on windows. >= 1.5 is supported on X11(Linux). Nothing for Apple.
Performance and Compatability Tips
- Tip: Avoiding relaunches - The first JVM used to parse the JNLP file might not satisfy the CLI args. If not download a new one
- Tip: Faster startup - use the update tag in the JNLP file to drastically reduce 2nd time startup (it downloads the new version and caches it in the background) The current run will use the version of the applet already in the cache but next time the newer version will be loaded. Downloads are done in the background
- Tip: Maintaining backward compatability - You can author content which runs on earlier versions of the plugin but which takes advantage of the new features. Simply provide a fully-formed applet tag as well as the jnlp_href parameter. Earlier versions will ignore the jnlp_href parameter. The new plugin will ignore the archive and code parameters
- Tip: Using JNLP extensions. It is now trivial to use extensions (e.g. OpenGL) which are available to the desktop. Use OpenGL via JOGL, OpenAL via JOAL, plus JavaFX player components such as webkit, video, and scene graph. Simply refer to the extension JNLP in the resources section of your JNLP file
Branding
- You can now replace the standard Java logo with an animated GIF. This is centred in the middle of the applet. There is also better control over foreground and background colours
- Can still do better. Phase the app as a tiny app first, then download the rest behind it (using the JNLP download services.) You can use this to draw a progress indicator
Java/Javascript Integration
- The bridge has been re written. It has been made as compatible as possible with previous implicit LiveConnect spec from Mozilla (in association with Mozilla)
- It is essentially as fast in all cases as the previous applet plugin
- You can now call static java methods in your applet code from javascript. e.g. new JFrame and launch a new JFrame window. NOTE: Can't do this in flash
- All this functionality is implemented on all browsers
- It is also multithread safe ("It will not crash the broowser")
- There is in addition a unified security model - this allows you to call from javascript to java. Security wise, this treats the JavaScript like an unsigned applet coming from the origin domain of the web page (which may not be the same as the applet). This is now done on all browsers. This does break signed JavaScript if anyne is using it (but Mozilla thought no-one used it)
- You can pass a JavaScript array into Java and it gets converted automagically.
- Overloaded methods are supported
- There is a new keyword "packages" which gives you access to the java namespace from JavaScript. This means you can execute static methods etc. This means the functions from your applet are extended into the surrounding web page
- Java is Multithreaded but JavaScript is not. How do you reconcile this? There is a per-applet worker thread which handles JavaScript to Java calls. It is distinct from all the other Java threads in your applet (e.g. AWT Dispatch thread etc.) If you call into the applet from JavaScript, this will do the processing
Manipulating the DOM
- Your applet can now manipulate the page on which it lives
- But the W3C DOM access has been sketchy support until now.
- Use org.w3c.dom.Document. Can now get the document from java code. com.sun.java.browser.plugin2.DOM.getDocument is a nice way to do this in Java code
Access to Web Services
- Flash does not have as rich a security architecture
- Cross-Domain XML standards specify how untrusted code may gain access to arbitrary web services on the internet (c.f. http://crossdomainxml.org)
- Now you can access all the web services which provide services via this spec. e.g. Amazon, Facebook, Flickr, picase, etc.
- BUT Domain restrictions are not supported
Experimental Functionality
- JavaFX applets. You can now drag and drop an applet out of the browser. Even if you shut down the browser it will still run
- Stress: This is OPTIONAL. You must state that you want to switch this feature on
- Consequently the Applet is now a mini app which runs everywhere. Sun have bridged the in- and out-of- browser deployment gap
- You turn this on with the "draggable" parameter. You then simply Alt-left click and drag. (the key binding is customisable). By default the new frame is undecorated and opaque
- A seperate "close" button is created by default. (Again this is customisable). You as a developer provide a seperate method called setAppletCloseListener. If this is implemented, some sort of close button will be provided in your applet. This is called when you shut the applet down
- While the page the applet came from is still visible you can still talk to the page. When the page goes away, the applet is dynamically morphed into a JWS application. NOTE: You can't talk to the page you came from, but you can still load a URL. Therefore services degrade very gracefully (JavaScript calls will error etc.). When your applet is closed, the normal applet shutdown process is executed
Questions
- What about older browsers and JREs? - The team have worked closely with Mozilla to massively simplifythings. As a result 90,000 lines of code thrown out from the browser (glue code). Unfortunately they can't back port it to Firefox 2 but Firefox 3 is supported. IE 5 and 6 are also supported.
- Who takes care of the 1.4 and 1.5 experience? - There is a new deployment toolkit which will help with the graceful degradation of functionality
- What about JOGL; do you still have to jump through security hoops like you used to? - With the new seperate JVM launcher (with your params) this is really easy. If the user hasn't seen the certificate JOGL requires to run, they will see a dialog and be asked to trust it
- What about 64 bit on Linux? - Sun are working on a beta program now. The binaries are provided to the windows platform now. The next step is Linux (Expect something beta in the next few months)
- Mac story? - no comments on a Mac port at this time. thankyou
- Can I specify update version of JVM? - yes, in the JNLP file, but there are issues with auto downloading pre-release versions of the JRE. However, if I download the JRE/SDK then the plugin comes down and is on by default
- Is there an email to hit with questions?: kenneth.russell@sun.com
- Java Web Start used to behave v. differently from Applets in many ways. Have you fixed it? - Yes. Multiple signers from multiple files are now possible as long as you split them across multiple JNLP files
- Can I detect that I have dragged an applet off a page and subsequently the page is reloaded so I can make sure only one applet is running?: - Yes. The JNLP will check and if one is already started the start in the page will fail
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
- Ken Russell & Sven Goethel
Intro
- Applets in Java6_u10 is a ground up re-write
- Applets no longer ru the browser process. (in the classic java browser plugin all the applets ran in same VM in the browser process)
- Now there is a thin JVM broker in the browser process and you can launch a new (seperate) VM per applet when required
- Advantages: Improved reliability (no browser crashes), no browser hang when loading, built in JNLP support (specify command line args on per applet instance), improved java/javascripts bridge and improved Vista support (signed applets run in protected mode IE)
Deploying via JNLP
- You can launch applets directly from JNLP files
- These follow the same descriptor as those in the Java Web Start architecture (therefore you can run your app as an applet or JWS application)
- Referencing resources - You can also reference your resources using relative (rather than absolute as with the old plugin) code paths. This allows simple migration of content from server to server and place to place
- There is also a new applet-desc tag. This references the main class, width, height
- There is also an information tag and a resources tag which references the other jar files providing info info like eager loading etc.
Command Line Arguments
- You can now define the CL args used to start the applet's JVM such as heap size, GC type, JRE version, plus properties - e.g. if you want to start an OpenGL applet
- The plugin launches a new JVM machine instance when necessary (separate_jvm tag)
- Demo: Quake 2 now runs in an applet - this shows the isolation and seperate JVM features
- The end user can specify CLIs too. There is a secure and a non-secure set.
- You must sign your jar and jnlp descriptor to get the capability to get an unrestricted JVM for your app
JRE Version Selection
- This allows you to run a particular applet in a particular version of the JRE
- This is useful for enterprises who want to QA applets against a particular JRE version
- It uses 'plus' semantics so you can have a JRE or later, or a specific family (1.4, 1.5 etc.)
- Auto download of requried but not installed JREs is supported by JNLP - you can also reference an installer for JWS
- Versions?: As early as 1.4.2 is supported on windows. >= 1.5 is supported on X11(Linux). Nothing for Apple.
Performance and Compatability Tips
- Tip: Avoiding relaunches - The first JVM used to parse the JNLP file might not satisfy the CLI args. If not download a new one
- Tip: Faster startup - use the update tag in the JNLP file to drastically reduce 2nd time startup (it downloads the new version and caches it in the background) The current run will use the version of the applet already in the cache but next time the newer version will be loaded. Downloads are done in the background
- Tip: Maintaining backward compatability - You can author content which runs on earlier versions of the plugin but which takes advantage of the new features. Simply provide a fully-formed applet tag as well as the jnlp_href parameter. Earlier versions will ignore the jnlp_href parameter. The new plugin will ignore the archive and code parameters
- Tip: Using JNLP extensions. It is now trivial to use extensions (e.g. OpenGL) which are available to the desktop. Use OpenGL via JOGL, OpenAL via JOAL, plus JavaFX player components such as webkit, video, and scene graph. Simply refer to the extension JNLP in the resources section of your JNLP file
Branding
- You can now replace the standard Java logo with an animated GIF. This is centred in the middle of the applet. There is also better control over foreground and background colours
- Can still do better. Phase the app as a tiny app first, then download the rest behind it (using the JNLP download services.) You can use this to draw a progress indicator
Java/Javascript Integration
- The bridge has been re written. It has been made as compatible as possible with previous implicit LiveConnect spec from Mozilla (in association with Mozilla)
- It is essentially as fast in all cases as the previous applet plugin
- You can now call static java methods in your applet code from javascript. e.g. new JFrame and launch a new JFrame window. NOTE: Can't do this in flash
- All this functionality is implemented on all browsers
- It is also multithread safe ("It will not crash the broowser")
- There is in addition a unified security model - this allows you to call from javascript to java. Security wise, this treats the JavaScript like an unsigned applet coming from the origin domain of the web page (which may not be the same as the applet). This is now done on all browsers. This does break signed JavaScript if anyne is using it (but Mozilla thought no-one used it)
- You can pass a JavaScript array into Java and it gets converted automagically.
- Overloaded methods are supported
- There is a new keyword "packages" which gives you access to the java namespace from JavaScript. This means you can execute static methods etc. This means the functions from your applet are extended into the surrounding web page
- Java is Multithreaded but JavaScript is not. How do you reconcile this? There is a per-applet worker thread which handles JavaScript to Java calls. It is distinct from all the other Java threads in your applet (e.g. AWT Dispatch thread etc.) If you call into the applet from JavaScript, this will do the processing
Manipulating the DOM
- Your applet can now manipulate the page on which it lives
- But the W3C DOM access has been sketchy support until now.
- Use org.w3c.dom.Document. Can now get the document from java code. com.sun.java.browser.plugin2.DOM.getDocument is a nice way to do this in Java code
Access to Web Services
- Flash does not have as rich a security architecture
- Cross-Domain XML standards specify how untrusted code may gain access to arbitrary web services on the internet (c.f. http://crossdomainxml.org)
- Now you can access all the web services which provide services via this spec. e.g. Amazon, Facebook, Flickr, picase, etc.
- BUT Domain restrictions are not supported
Experimental Functionality
- JavaFX applets. You can now drag and drop an applet out of the browser. Even if you shut down the browser it will still run
- Stress: This is OPTIONAL. You must state that you want to switch this feature on
- Consequently the Applet is now a mini app which runs everywhere. Sun have bridged the in- and out-of- browser deployment gap
- You turn this on with the "draggable" parameter. You then simply Alt-left click and drag. (the key binding is customisable). By default the new frame is undecorated and opaque
- A seperate "close" button is created by default. (Again this is customisable). You as a developer provide a seperate method called setAppletCloseListener. If this is implemented, some sort of close button will be provided in your applet. This is called when you shut the applet down
- While the page the applet came from is still visible you can still talk to the page. When the page goes away, the applet is dynamically morphed into a JWS application. NOTE: You can't talk to the page you came from, but you can still load a URL. Therefore services degrade very gracefully (JavaScript calls will error etc.). When your applet is closed, the normal applet shutdown process is executed
Questions
- What about older browsers and JREs? - The team have worked closely with Mozilla to massively simplifythings. As a result 90,000 lines of code thrown out from the browser (glue code). Unfortunately they can't back port it to Firefox 2 but Firefox 3 is supported. IE 5 and 6 are also supported.
- Who takes care of the 1.4 and 1.5 experience? - There is a new deployment toolkit which will help with the graceful degradation of functionality
- What about JOGL; do you still have to jump through security hoops like you used to? - With the new seperate JVM launcher (with your params) this is really easy. If the user hasn't seen the certificate JOGL requires to run, they will see a dialog and be asked to trust it
- What about 64 bit on Linux? - Sun are working on a beta program now. The binaries are provided to the windows platform now. The next step is Linux (Expect something beta in the next few months)
- Mac story? - no comments on a Mac port at this time. thankyou
- Can I specify update version of JVM? - yes, in the JNLP file, but there are issues with auto downloading pre-release versions of the JRE. However, if I download the JRE/SDK then the plugin comes down and is on by default
- Is there an email to hit with questions?: kenneth.russell@sun.com
- Java Web Start used to behave v. differently from Applets in many ways. Have you fixed it? - Yes. Multiple signers from multiple files are now possible as long as you split them across multiple JNLP files
- Can I detect that I have dragged an applet off a page and subsequently the page is reloaded so I can make sure only one applet is running?: - Yes. The JNLP will check and if one is already started the start in the page will fail
Advanced Enterprise Debugging Techniques - TS-6072
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Neal Ford, Thoughtworks
Debugging 101
- Problem? Before you do anything else, step away from the computer and think. This is the hardest thing to learn
Forensic debugging with loggers
- Log4J
- Examine log4j's xml log files with Chainsaw (aSwing UI) which hacks up your log dumps. It allows filtering to look at particular log files post mortem
- You can create a genrator for Log4J which you can plug right into chainsaw which allows you to view running apps live (still with the filtering etc.)
Fiddling with your application in production? - Fiddle
- Ideally your debug (dev.) environment would like like production - but it never does
- But you don't want to turn on remote debugging in live
- There is an OSS app which allows you to fiddle with your app using scripting languages to find out what is going on in there in production
- Fiddle: A single jar file with no external dependencies which you drop into your app server
- Get it from fiddle.dev.java.net
- It provides a fiddle executable servlet. You send it some scripting code which it will execute and display the output on the fiddle web page
- It also comes with fiddler - a text area where you type in the script
- fiddle-scripts - pre rolled fiddle scripts for you to use
- Your scripts can use groovy, javascript, jython (scripting so you don't need to compile.)
- e.g. get info from system tables, get the loader path, find out which xml parser is loaded, get all the system properties, you can even tweak security settings and get values out of HTTP sessions
- To use it, take the fiddle servlet, put it behind a password protected page, and use as required
Debugging back in time - Omniscient Debugger
- Takes regular snapshots when your app runs and saves them.
- Allows you to visit any snapshots in the past - i.e. run your app until it gets corrupted, then you go back in time to find out where things went wrong. It effectievly sprinkles breakpoints and allows you to go and use them
- Now you can walk through, one snapshot at a time. You can also walk backwards through the timestamps.
- All you need to tweak is how granular you want the snapshots. But you can make this pretty coarse grained and still see how things work
- Good for multithreaded problems
- It is a jar file and easy to run along with your application server
- N.b. it uses a lot of memory but can save you time in debugging
Untangling Jars - Jar Analyzer
- It's hard to look at a directory with all the jars and see what all the relationships between them are
- use jar analyzer
- It has a xml output (ugly but packed with data)
- Fortunately this comes with a XSL stylesheet => transformed to a nice page.
- Points out cyclic dependencies
- There is also a graphical view output which gives a Tree view. The problem is this can get really complicated. All it does is show you the mess
- Commercial alternatives? - SonarJ. The Spring project used this to make sure they didn't get into such a mess. Now they use Structure101
- Step one is knowing that a mess exists. Then you can fix it by hand or use one of these tools
Mocks for Faster Debugging - MockRunner
- There are multiple testing levels: unit (simple, single component - my thing works like I think it does), functional (my stuff works with their stuff if their stuff isn't there yet - mocking. I can also mock out parts of J2EE), integration (major chunks talking to one another - my stuff and their stuff works together) and user acceptance
- To speed up functional tests you want a Mocking infrastructure (e.g. MockRunner to mock J2EE infrastrcture). You use it to ceate a test fixture (harness). This then rnus the tests using regular JMock tests
- Now it is almost instantaneous to run your tests. An example is to mock out JMS with thread pools. You can also do this with Servlets, EJB, and all the other interesting parts of J2EE
- Great time gains
Automating UAT - Selenium
- Use this to automate tests which run against your running web app for UAT
- It can also be used to automate tedious debugging tasks
- It includes a side project called Selenium IDE to record tests
- This allows you to automate debugging "wizard" style web applications and removes the hassle of manually stepping through the bits you have working to the broken bit in wizards
- Because you always think "this is the last time I have to do this" but it never is
- You don't even need Selenium installed to do it; it's a Firefox plugin. You can record your interactions with a web site and play it back automatically
- There are single step and pause buttons in Selenium IDE for stepping through
- All you do is record an interaction the first time you do it manually
- When you record your steps, it is stored as an HTML table. This looks like a fit test
- You can export this table to java - this is then a regular JUnit test case which you can run standalone
- You can also export as Ruby, Python, Perl and other languages
- Basically it's an interation API for a web browser/applications - you could even create a black box for web browsers (i.e. in airplanes) to track what users are doing.
- You could also get your QA dept to record their tests so they can send you the script which breaks the app
- What if I want to run in IE? => use Selenium remote control. I don't want to deploy parts of Selenuim to a production application. SRC creates a proxy server (flavour of jetty) which the test runs against which in turn runs against the real server by spawning a new instance of a browser. There is no selenium on the production server. You can tell it to launch IE, or some other browser.
- You can have a headless server which kicks off many different types of test clients and kick this off from your CI server
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Neal Ford, Thoughtworks
Debugging 101
- Problem? Before you do anything else, step away from the computer and think. This is the hardest thing to learn
Forensic debugging with loggers
- Log4J
- Examine log4j's xml log files with Chainsaw (aSwing UI) which hacks up your log dumps. It allows filtering to look at particular log files post mortem
- You can create a genrator for Log4J which you can plug right into chainsaw which allows you to view running apps live (still with the filtering etc.)
Fiddling with your application in production? - Fiddle
- Ideally your debug (dev.) environment would like like production - but it never does
- But you don't want to turn on remote debugging in live
- There is an OSS app which allows you to fiddle with your app using scripting languages to find out what is going on in there in production
- Fiddle: A single jar file with no external dependencies which you drop into your app server
- Get it from fiddle.dev.java.net
- It provides a fiddle executable servlet. You send it some scripting code which it will execute and display the output on the fiddle web page
- It also comes with fiddler - a text area where you type in the script
- fiddle-scripts - pre rolled fiddle scripts for you to use
- Your scripts can use groovy, javascript, jython (scripting so you don't need to compile.)
- e.g. get info from system tables, get the loader path, find out which xml parser is loaded, get all the system properties, you can even tweak security settings and get values out of HTTP sessions
- To use it, take the fiddle servlet, put it behind a password protected page, and use as required
Debugging back in time - Omniscient Debugger
- Takes regular snapshots when your app runs and saves them.
- Allows you to visit any snapshots in the past - i.e. run your app until it gets corrupted, then you go back in time to find out where things went wrong. It effectievly sprinkles breakpoints and allows you to go and use them
- Now you can walk through, one snapshot at a time. You can also walk backwards through the timestamps.
- All you need to tweak is how granular you want the snapshots. But you can make this pretty coarse grained and still see how things work
- Good for multithreaded problems
- It is a jar file and easy to run along with your application server
- N.b. it uses a lot of memory but can save you time in debugging
Untangling Jars - Jar Analyzer
- It's hard to look at a directory with all the jars and see what all the relationships between them are
- use jar analyzer
- It has a xml output (ugly but packed with data)
- Fortunately this comes with a XSL stylesheet => transformed to a nice page.
- Points out cyclic dependencies
- There is also a graphical view output which gives a Tree view. The problem is this can get really complicated. All it does is show you the mess
- Commercial alternatives? - SonarJ. The Spring project used this to make sure they didn't get into such a mess. Now they use Structure101
- Step one is knowing that a mess exists. Then you can fix it by hand or use one of these tools
Mocks for Faster Debugging - MockRunner
- There are multiple testing levels: unit (simple, single component - my thing works like I think it does), functional (my stuff works with their stuff if their stuff isn't there yet - mocking. I can also mock out parts of J2EE), integration (major chunks talking to one another - my stuff and their stuff works together) and user acceptance
- To speed up functional tests you want a Mocking infrastructure (e.g. MockRunner to mock J2EE infrastrcture). You use it to ceate a test fixture (harness). This then rnus the tests using regular JMock tests
- Now it is almost instantaneous to run your tests. An example is to mock out JMS with thread pools. You can also do this with Servlets, EJB, and all the other interesting parts of J2EE
- Great time gains
Automating UAT - Selenium
- Use this to automate tests which run against your running web app for UAT
- It can also be used to automate tedious debugging tasks
- It includes a side project called Selenium IDE to record tests
- This allows you to automate debugging "wizard" style web applications and removes the hassle of manually stepping through the bits you have working to the broken bit in wizards
- Because you always think "this is the last time I have to do this" but it never is
- You don't even need Selenium installed to do it; it's a Firefox plugin. You can record your interactions with a web site and play it back automatically
- There are single step and pause buttons in Selenium IDE for stepping through
- All you do is record an interaction the first time you do it manually
- When you record your steps, it is stored as an HTML table. This looks like a fit test
- You can export this table to java - this is then a regular JUnit test case which you can run standalone
- You can also export as Ruby, Python, Perl and other languages
- Basically it's an interation API for a web browser/applications - you could even create a black box for web browsers (i.e. in airplanes) to track what users are doing.
- You could also get your QA dept to record their tests so they can send you the script which breaks the app
- What if I want to run in IE? => use Selenium remote control. I don't want to deploy parts of Selenuim to a production application. SRC creates a proxy server (flavour of jetty) which the test runs against which in turn runs against the real server by spawning a new instance of a browser. There is no selenium on the production server. You can tell it to launch IE, or some other browser.
- You can have a headless server which kicks off many different types of test clients and kick this off from your CI server
Groovy: The Red Pill (or Metaprogramming : The way to blow the mind of a Buttoned-Down Java Developer) - TS-5572
DISCLAIMER:
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Scott Davis
- http://aboutGroovy.com
- Writing mastering Grails for IBM
Groovy + Java
- Groovy is a dynamic language that runs on the JVM
- Since it is implemented in Java, the two sesmlessly integrate - this is not a "burn the boats" migration
- You can take almost any .java file and rename it .groovy and chances are it will run (though it won't be idiomatic groovy)
- Therefore I can still be a "Java Programmer" - the semantic mis-match is kept to a bare minumum
- As you learn the Groovy idioms, you can get into it slowly
Installing Groovy
- Three steps (Dead easy)
- To add Groovy support to your existing java project you just add the single groovy-all-1.5.4.jar (c.f. spring, hibernate, etc.)
HelloJava == HelloGroovy
- println "Hello Groovy World"
- The Groovy code drops all the need for "public static void main..."
- Compile with grvyc => .class files are output
- Run javap against the class files and we see how the simple groovy code gets to a class file
- Groovy offers optional parentheses and semi-colons
- Groovy supports method pointers
- Groovy == Java
Method Pointers
- say = System.out&println # => Pointer to the method
- say "Hola" # => Hola
- Groovy also supports operator overloading (e.g. '<<' to append to an array or other collection) - load = songs.&add ; load "Yellow Submarine" # => makes "load" a pointer to the '.add' method
- play = System.out&println # => ditto
POGOs
- Plain old groovy object - compiles to a class which extends java.lang.Object
- Drop in replacement for a POJO
- semicolons and mutators (get/set) are missing but then the compiler can generate it for you (Note: this is not in-IDE generation; the code is simply not there. It does not need to be there)
- No returns - the last line of a method is an implicit return (though you can add it if you wish)
- No accessors. All classes and methods are public by default, all attributes are private by default
- Strings: "${title} by ${artist}" # => GStrings which support native string interpolation (c.f. Ant, JSPs, Ruby)
- Vararg, named arg constructors: load new Song(title:"Revolution", artist:"The Beatles") ; load new Song(artist:"Pearl Jam", title:"Alive")
Closures
- Definiton: Hunk of exec code which does not have a corrsponding class (Java you must have an anonymous, inner class)
- hi = {println "Hola"} ; hi() # => Hola
- The ExpandoMetaClass depends on this - it adds methods to classes on the fly and so you need free floating methods
- Groovy Shell (c.f. IRB): groovysh:00> 7.times{ println "Hi"} # => 'times' is a closure bolted onto the 7 which is a java.lang.Integer
- d = new Date() ; d++ ; d++ # => adds a day to d
Metaprogramming
- Dynamically adding new methods on the fly at runtime
- message = "Good Moring, JavaOne"; println message
- message = "Good Moring, JavaOne"; println message.shout # method missing exception. we need to add this method to String
- message = "Good Moring, JavaOne"; String.metaClass.shout = {-> delegate.toUpperCase() }; println message # => 'GOOD MORNING, JAVAONE' We added a new method to String
- How does this work? - the ExpandoMetaClass at the core of groovy is where all method calls hit first before hitting the actual classes. Therefore we can do stuff like adding methods to final classes like String
- message = "Good Moring, JavaOne"; String.metaClass.toUpperCase = {-> delegate.toLowerCase() }; println message # => 'good morning, javaone' We overloaded an existing method on String. This is useful in unit testing because we can re route method calls
- Want to give it a go? Start with it in a unit testing environment. It's a gateway drug. ;-)
- class Ipod { def songs = [] ; ipod() { songs <<> println song }; songs.size() } } # 'def' means I don't care what type it is.
- the 'each' iterator. It's all over the place so use it
- family = [dad:"Chris", mom:"Amanda"] # => java.util.HashMap
- family.each{println it} # => Chris, Amanda # 'it' is like the current pointer from the iterator. You can rename it. c.f. Ruby: family.each do |it| it.println end
- family.mom # => Amanda
Groovy Unit Testing
- class IpodTest extend GroovyTestCase { void testPlay() { def expected = 1 ; assertEquals expected, i.play ; } }
Operator Overloading
- You can do it
- Good for DSLs
InvokeMethod and MethodMissing
- Allows you to call methods which don't existin until you call them
- Grails has this feature. It implements invokeMethod (called every time) and methodMissing (called first time)
- Therefore the methods are created when you want them
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
Scott Davis
- http://aboutGroovy.com
- Writing mastering Grails for IBM
Groovy + Java
- Groovy is a dynamic language that runs on the JVM
- Since it is implemented in Java, the two sesmlessly integrate - this is not a "burn the boats" migration
- You can take almost any .java file and rename it .groovy and chances are it will run (though it won't be idiomatic groovy)
- Therefore I can still be a "Java Programmer" - the semantic mis-match is kept to a bare minumum
- As you learn the Groovy idioms, you can get into it slowly
Installing Groovy
- Three steps (Dead easy)
- To add Groovy support to your existing java project you just add the single groovy-all-1.5.4.jar (c.f. spring, hibernate, etc.)
HelloJava == HelloGroovy
- println "Hello Groovy World"
- The Groovy code drops all the need for "public static void main..."
- Compile with grvyc => .class files are output
- Run javap against the class files and we see how the simple groovy code gets to a class file
- Groovy offers optional parentheses and semi-colons
- Groovy supports method pointers
- Groovy == Java
Method Pointers
- say = System.out&println # => Pointer to the method
- say "Hola" # => Hola
- Groovy also supports operator overloading (e.g. '<<' to append to an array or other collection) - load = songs.&add ; load "Yellow Submarine" # => makes "load" a pointer to the '.add' method
- play = System.out&println # => ditto
POGOs
- Plain old groovy object - compiles to a class which extends java.lang.Object
- Drop in replacement for a POJO
- semicolons and mutators (get/set) are missing but then the compiler can generate it for you (Note: this is not in-IDE generation; the code is simply not there. It does not need to be there)
- No returns - the last line of a method is an implicit return (though you can add it if you wish)
- No accessors. All classes and methods are public by default, all attributes are private by default
- Strings: "${title} by ${artist}" # => GStrings which support native string interpolation (c.f. Ant, JSPs, Ruby)
- Vararg, named arg constructors: load new Song(title:"Revolution", artist:"The Beatles") ; load new Song(artist:"Pearl Jam", title:"Alive")
Closures
- Definiton: Hunk of exec code which does not have a corrsponding class (Java you must have an anonymous, inner class)
- hi = {println "Hola"} ; hi() # => Hola
- The ExpandoMetaClass depends on this - it adds methods to classes on the fly and so you need free floating methods
- Groovy Shell (c.f. IRB): groovysh:00> 7.times{ println "Hi"} # => 'times' is a closure bolted onto the 7 which is a java.lang.Integer
- d = new Date() ; d++ ; d++ # => adds a day to d
Metaprogramming
- Dynamically adding new methods on the fly at runtime
- message = "Good Moring, JavaOne"; println message
- message = "Good Moring, JavaOne"; println message.shout # method missing exception. we need to add this method to String
- message = "Good Moring, JavaOne"; String.metaClass.shout = {-> delegate.toUpperCase() }; println message # => 'GOOD MORNING, JAVAONE' We added a new method to String
- How does this work? - the ExpandoMetaClass at the core of groovy is where all method calls hit first before hitting the actual classes. Therefore we can do stuff like adding methods to final classes like String
- message = "Good Moring, JavaOne"; String.metaClass.toUpperCase = {-> delegate.toLowerCase() }; println message # => 'good morning, javaone' We overloaded an existing method on String. This is useful in unit testing because we can re route method calls
- Want to give it a go? Start with it in a unit testing environment. It's a gateway drug. ;-)
- class Ipod { def songs = [] ; ipod() { songs <<> println song }; songs.size() } } # 'def' means I don't care what type it is.
- the 'each' iterator. It's all over the place so use it
- family = [dad:"Chris", mom:"Amanda"] # => java.util.HashMap
- family.each{println it} # => Chris, Amanda # 'it' is like the current pointer from the iterator. You can rename it. c.f. Ruby: family.each do |it| it.println end
- family.mom # => Amanda
Groovy Unit Testing
- class IpodTest extend GroovyTestCase { void testPlay() { def expected = 1 ; assertEquals expected, i.play ; } }
Operator Overloading
- You can do it
- Good for DSLs
InvokeMethod and MethodMissing
- Allows you to call methods which don't existin until you call them
- Grails has this feature. It implements invokeMethod (called every time) and methodMissing (called first time)
- Therefore the methods are created when you want them
Wednesday, May 07, 2008
Practical Applications of Static Bytecode Analysis and Transformation for the Java Platform - BOF-5839
DISCLAIMER
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
The JVM
- proven and reliable
- built for statically typed java language
- but now > 200 languages, which can be compiled to bytecode
- Classes can be generated and analysed at rnutime and compile time
The Class File Format
- Constant pool - field and method names, ty descriptors, string litersla
- Attributes - fields, methods, code
- debug information (line numbers etc, local variable names)
- exceptions
- User defined attributes
Class File Access and Modification Problems
- Lots of serialization and deserialization details
- constant pool management - managing constant pool indexes, references, missing or unised constants
- jump offsets - calculation of instruction offsets, inserting and removing instructions from the method
- Computation of stack size and StackMapTable - requires a control flow analysis
There are several bytecode frameworks to help hide the details - ASM is an example
ASM Bytecode Framework (v3.1)
- From France Telecom in 2002
- Initial goal: Fast tool to do simple bytecode transformations
- Core - generate classes, basic transforms
- Tree and analysis - in memory representation, analysis algorithms
- Commons - renaming, advice adaptor, inline JSR/RET subroutines, sort local variables, calculate serialVersionUID etc.
- Utilities - checker, decompiler, ASMifier (write code, compile to bytecode, then run ASMifier to see what ASM to write to create this)
"Visitor" Pattern Based
- Heavy use of the visitor pattern
- ClassReader implements. Takes a serialised input
- MyClassAdapter - can be chained to add, remove, change etc. things. In this respect this is very similar to SAX API
- ClassWriter implements. Writes serialized output
How Cobertura Uses ASM
- Instruments compiled classes
- Adds callbacks to the recording engine - for each method line, for each branching instruction
- Saves information about added callbacks
- Recording engine is called during test execution - records number of invocations for each callback to mark code branhes as tested. compares which code branhes have been tested with what hasn't been tested.
Google Singleton Detector
- Singletons are bad and make testing hard
- When you come to a project you want to get an overview - where is all the hairy stuff? Do it with ASM, based on heuristics (private constructor, hingleton (helper class enforcing a singleton), mingleton (class with static method that returns some state and do not have parameters), fingleton (public static field singleton) - all are ways to hide global state)
- Find with ClassVisitor (iterate over all classes in jar), MethodVisitor (mingletons), FieldVisitor (fingletons - look for global state)
- They tested Catalina - try and run two instances of Tomcat in same JVM on different ports and there will be shared state problems.
- Second tool - testability explorer - looks for global mutable state (just bad) and the lack of injectability (hard to test individual pieces in isolation). Assigns points to classes (goodness (DependencyInjection, interface / implementation seperation) and badness (global state))
- http://testabilityexplorer.org & on code.google.com/p/testability-explorer
- Take these ideas, put in a jar and add to CI when something hard to test is added
- It tells you the line number where a variable is not injected ("why I am marking this class hard to test")
- It tells you how easy (or hard) a project is to unit test. Not how covered it is
These are the tidied up notes I took from the session at the JavaOne 2008 conference in San Francisco, California. There may well be mistakes and ommissions. I will come back and correct them once the conference has completed. However my first priority is to get the information published before the backlog gets too large and swamps me. All comments welcome. Enjoy!
The JVM
- proven and reliable
- built for statically typed java language
- but now > 200 languages, which can be compiled to bytecode
- Classes can be generated and analysed at rnutime and compile time
The Class File Format
- Constant pool - field and method names, ty descriptors, string litersla
- Attributes - fields, methods, code
- debug information (line numbers etc, local variable names)
- exceptions
- User defined attributes
Class File Access and Modification Problems
- Lots of serialization and deserialization details
- constant pool management - managing constant pool indexes, references, missing or unised constants
- jump offsets - calculation of instruction offsets, inserting and removing instructions from the method
- Computation of stack size and StackMapTable - requires a control flow analysis
There are several bytecode frameworks to help hide the details - ASM is an example
ASM Bytecode Framework (v3.1)
- From France Telecom in 2002
- Initial goal: Fast tool to do simple bytecode transformations
- Core - generate classes, basic transforms
- Tree and analysis - in memory representation, analysis algorithms
- Commons - renaming, advice adaptor, inline JSR/RET subroutines, sort local variables, calculate serialVersionUID etc.
- Utilities - checker, decompiler, ASMifier (write code, compile to bytecode, then run ASMifier to see what ASM to write to create this)
"Visitor" Pattern Based
- Heavy use of the visitor pattern
- ClassReader implements. Takes a serialised input
- MyClassAdapter - can be chained to add, remove, change etc. things. In this respect this is very similar to SAX API
- ClassWriter implements. Writes serialized output
How Cobertura Uses ASM
- Instruments compiled classes
- Adds callbacks to the recording engine - for each method line, for each branching instruction
- Saves information about added callbacks
- Recording engine is called during test execution - records number of invocations for each callback to mark code branhes as tested. compares which code branhes have been tested with what hasn't been tested.
Google Singleton Detector
- Singletons are bad and make testing hard
- When you come to a project you want to get an overview - where is all the hairy stuff? Do it with ASM, based on heuristics (private constructor, hingleton (helper class enforcing a singleton), mingleton (class with static method that returns some state and do not have parameters), fingleton (public static field singleton) - all are ways to hide global state)
- Find with ClassVisitor (iterate over all classes in jar), MethodVisitor (mingletons), FieldVisitor (fingletons - look for global state)
- They tested Catalina - try and run two instances of Tomcat in same JVM on different ports and there will be shared state problems.
- Second tool - testability explorer - looks for global mutable state (just bad) and the lack of injectability (hard to test individual pieces in isolation). Assigns points to classes (goodness (DependencyInjection, interface / implementation seperation) and badness (global state))
- http://testabilityexplorer.org & on code.google.com/p/testability-explorer
- Take these ideas, put in a jar and add to CI when something hard to test is added
- It tells you the line number where a variable is not injected ("why I am marking this class hard to test")
- It tells you how easy (or hard) a project is to unit test. Not how covered it is
Subscribe to:
Posts (Atom)
View Andrew Law's profile