Book: “Dynamics of Software Development”

Image “Dynamics of Software Development” by Jim McCarthy

Although this book published almost 20 years ago, it has some timeless rules that are still applicable today. Here are a few lines from the book:

  • “If the leader can then resonate with the team’s complex emotional state–identify with it, articulate it, and give the whole constellation of feeling and thought a visible, concrete reality in his or her own personal voice or gesture–the boundaries among the individual team members and between the team members and the leader will collapse.”
  • “The truth is hard to take, challenges your character, but whether the truth is offered by someone else or you discover it independently, you have to listen to it eventually… Not only must you listen to the truth, but you must broadcast it to the rest of the team. And the medium for transmitting truth among humans, like it or not, is emotion.”
  • “not to slow the pace of change, or to create more stability; but to get good at change, at managing technology in motion.”
  • “you are not going to get away with many more than two developers for every one QA person”
  • “If a balanced group of people are mutually accountable for all aspects of design, development, debugging, QA, shipping, and so on, they will devise ways to share critical observations with one another. Because they are accountable, if they perceive it, they own it. They must pass the perception to the rest of the team.”
  • “leadership in software development require a high degree of sensitivity to the human nature of the enterprise, an awareness of the underlying drives and emotions that determine the team’s behavior.”
  • “if you are having a hard time understanding something about the team, you can look to the software. If the team and the software both tell you the same thing, you can act on it with some degree of confidence. Conversely, if the software hasn’t reach the desired state, the way to fix it is to analyze the genesis of the problem in the team.”
  • “Freedom is the cornerstone of empowerment, freedom to develop and apply judgement, freedom to think and say what needs thinking and saying, freedom to take risks without extraneously punitive consequences.”
  • “Empowerment is the result of teaching and learning, not neglect and anarchy.”
  • “invest significantly in paradigm-shifting features… Once you make a breakthrough in the paradigm arena, your competitor will be forced back to the drawing board, even if he or she is momentarily ahead in the feature shoot-out.”
  • “Shipping is the hardest thing to do. If you’re better at shipping than y our competitor, you’re likely to be better than y our competitor at virtually everything. Timely, frequent shipping is the manifestation of well-being on a software development team.”
  • “Teaching becomes the primary function of leaders and managers.”
  • “in a product that had unity, each element would be essential to the value of the whole and all essential elements would be there… since everything the customer needed would be there, the customer wouldn’t be tempted to go beyond the present experience, and that since nothing would be there that wasn’t required, the customer’s absorption into the world of the product wouldn’t be disturbed.”
  • “It should be a fundamental dogma that the person who has to do the work should predict the amount of time it will take.” “The ultimate act of disempowerment is to take away the responsibility for the schedule from those who must live by it.”
  • “You need to build schedule meticulously from the bottom up. Each person who has a task to do must own the design and the execution of the task and must be held accountable for its timely achievement. Accountability is the twin of empowerment. The two together can create a reasonable software development plan.”
  • “As a development manager, you’re working with only three things: resource (people and money), features (the product and its quality), and the schedule.” “When considering the possible solution to a schedule shortfall, keep in mind there are only four possible: add time, subtract feature, add resources, or do some combination of the three.”
  • “when something is unknown, the best policy is to state that simple fact, even if the unknown is not knowing when the software will slip.”
  • “The goal on a software development project is not to have the correct plan in advance but to make the right decision every day as things that were unknown become known.”
  • Team interdependence as a motivational factor to deliver software on-time: “The goal is to create a network of self-motivated individual commitments.”
  • “If you have an ’empowered development team’ deliverables are negotiated among developers, writers, program managers, and testers. ‘Management’ has virtually nothing to say about deliverables.”
  • “Slipping isn’t the problem. Being surprised by slipping is the problem. A slip doesn’t say that the product is too hard to develop. Being surprised by a slip says that the organization is broken. People aren’t thinking. People aren’t talking to each other. People aren’t aware of the global situation.”
  • “A good general rule is that you don’t reset the schedule until the total extent of the slip is known for each component, the cause of the slip are understood, and remedies are in place.”
  • “The biggest mistake I see managers make as they hire people for software development team is that they overvalue a particular technical skill… Much more important than a particular technical skill is a history of relevant skills accumulation.”

Book: “Notes to a Software Team Leader”

Image“Notes to a Software Team Leader” by Roy Osherove is a refreshing read. I like his way of explaining concepts, making them easy to understand. His perspective that “a team leader grows the people in their team” is inspirational. Here are a few lines from the book:

  • “To get out of survival mode, you have to worry about one thing: creating slack time as a standard in your work process.” This is necessary for the team to find the time they need to acquire new skills, knowledge and capabilities to better handle challenges.
  • “To get to the next stage and to become much better than you are, you have to let go of the things you already know. You have to let go of the safety of the current position you’re in, so that you climb to the next level.” The concept of letting go what you already know is a new concept to me.
  •  “What’s under their control? Usually, their time and what they choose to work on are the only things under their administrative control.” This means that an engineer can’t reliably guarantee that s/he will be able to fix a bug by tomorrow. Instead s/he can commit to spending at least 6 hours today and 6 more hours tomorrow to work on fixing the bug.
  • “What are you going to do about it?” This is a question a leader can challenge his/her direct reports to learn how to handle a challenge by him/her self and commit to do something about it.
  • Forces that can influence behavior change: personal ability, personal motivation, social ability, social motivation, environmental ability and environmental motivation. Personal ability and motivation are straight forward. Social ability and motivation covers people around the person that is attempting to make a behavior change. Environmental ability and motivation extends not only the physical environment, also the organizational environment that controls the pay, bonus, etc.
  • “Team leadership is about creating an environment in which everyone can flourish to the best of each person’s ability–including yours.”

Notes about PersistJS

After I tried to use PersistJS, I am impressed. Then people are asking me to write-up my findings. So, here are my notes about PersistJS:

The first recommended reading is Why This is Awesome section. It makes a good case why you should use PersistJS. One interesting aspect of PersistJS is explained in its README: “Each backend is wrapped by PersistJS and exploses the exact same interface, which means you don’t have to know or care which backend is being used.” This means that you are not limited by a single technology and this enables you to adopt to a different storage easily. This statement might leave you with the question about which one does PersistJS choose? To answer this question, I found the order which PersistJS chooses a back-end store:

  1. gears
  2. localstorage
  3. whatwg_db
  4. globalstorage
  5. flash
  6. ie
  7. cookie

In my first implementation that uses PersistJS, I did not load extras/gears_init.js or extras/swfobject.js, this means that google gears and flash back-end stores are not enabled in my use of PerisstJS. With this setup, localstorage becomes the first one that persist JS attempt to store data.

PersistJS uses HTML 5 localStorage DOM attribute to implement its localstorage storage. HTML 5 is a new specification in draft state now. localStorage is one of the two storage mechanisms introduced, which “is designed for storage that spans multiple windows, and lasts beyond the current session.” The the storage format that HTML 5 exposes are key/value pairs. Both keys and values have to be strings. For further reading, see specification draft.

If you are an Ext JS user, you can use PersistStateProvider to persist UI state and application data.

A quick scan of “The Future of Programming Environments” article

Andreas Zeller‘s “The Future of Programming Environments”, provides an high level overview on the current trends of development tools. In this article, Zeller stated that development tools are evolving from extensible frameworks that supports integrated tools and their automation to “automated assistance in all development decisions”. Zeller further pointed out many known techniques that collects data about a piece of code in both static form and during run-time, then apply various automated analysis techniques that leads to visualization, statistics, data mining and machine learning to increase the productivity and code quality.

The paper serve more of a “review” or a “digest” of current trend of tools development than a “research” paper. Zeller certainly stated various recommendations that can serve as “best practices” for developers of tools. I would treat these recommendations as more of “common sense” than “design patterns”. This certainly is a good introductory paper with some interesting ideas and examples on what others have done in the tools space.

NetBeans 6 beta 1 is more JavaScript friendly than earlier releases

I’ve been using NetBeans 6 beta 1 for a while now. Today I just found out that if I am working on a static file such as a JavaScript file, all I need to do is to redeploy the project to the local server instance to see the changes. No need to rebuild or recompile. This is very cool. I don’t have to wait for the compilation of all that Java code in the same project to see changes in JavaScripts. This saves me a lot of time and I don’t have to go to a command line and do the copy myself. Again, very cool!

First impression on package mangement on Mac OS X

Ok. I’ve only used a Mac for a couple days, they were certainly long days with many steep learning experiences. Here are some of my comments:

On Mac OS X, you have several options to install an application. You can use pre-packaged binaries such as Universal Binary or a platform specific installable binary for your Intel or PowerPC based Mac computers. This class of binary is equivalent to setup.exe/intall.exe on Windows except this is much better at the first glance. From the user point of view the installation process typically involves double click or click OK/Yes until you are done, which translates to about 3-4 times and this is much shorter than most other installation steps that I am used to. To me, the impressive aspect of installing applications from pre-packaged binaries is the concept of drag-and-drop of a single icon into just about anywhere you want to install the application. Behind that, there are a lot of other complexities that is being abstracted out for the user. That level of simplification is an eye opening experience for me.

Working with installable binary directly by point-and-click your way through essentially means you are managing the application portfolios by you self. This reminded me of early Fedora days when I would go to different application repositories or web sites to download individual rpm packages. Having to manage individual packages you self can become a challenging task. At least, you may loose track of which packages are getting outdated and need to be updated. In the worse scenario, you may need to figure out all the dependencies you self and install them in the correct order. This certainly isn’t the most optimal way to manage a modern operating system. Even with extreme ease of use from UI point of view, I still have to do all the clicking on each package isn’t all that exciting.

Besides the regular installable binary, you can use a package management system, Fink. It provides both a repository for many open source software and a popular command line tool, apt-get, to easily manage the packages. apt-get is a very popular tool among Linux distributions and it made me feel at ease initially. What’s unusual to me about Fink is that it isn’t just a binary package management system, it can even handle install from source code, similar to how the emerge tool from Gentoo. You can configure it to use newer unstable version of the source tree and with a single line of command, you can essentially trigger the system to download the source package from the repository, compile and install the compiled binary in a sequence that satisfies all dependencies. In addition, all software are installed in an isolated directory so they do not conflict with any existing software you may already have. This essentially is the same approach as‘s way of manage applications. Fink combines many great designs and features among many modern package management systems and it certainly looks great on paper.

In practice, I found it also suffers similar issues as emerge on Gentoo when you venture beyond the stable binary into unstable source packages. A software package may fail to build. If a such error does occur, the user may have do some heavy-weight lifting to troubleshoot the error. I did tried to build Gnucash and it failed on me. One big difference that I found between Fink and Gentoo’s emerge it it’s community. When I encountered an error when attempting to install or upgrade a package, all I had to do was go to Gentoo’s we forum and past the error message. Most of the time, I would see people’s discussion about that specific error. Many would post suggestions and solutions. Having such “howto” materials available EASILY was really useful, important, and educational to me. on the other hand, my first impression with Fink when venture out to the unstable tree has left me stranded for several days so far. It appears to me that the community activities among Fink circles wasn’t as active as other forums among major Linux distributions that I’ve worked with. So, be careful when you are thinking of turn that switch to unstable tree. I would advice you to do some search for support materials first.

In addition to Fink, you also have the option of Macports, which is a source based package management system. It requires you to download and install Xcode, a free development tool from Apple. Xcode basicly contains gcc compilers, an IDE and utilities that are geared toward developers to write applications for Mac OS X. At about 923MB download size, Xcode certainly makes Macports a much more heavyweight than other package management options. Macports works similar to Gentoo’s emerge command, except you use port command to install applications. One thing typical of a source based package management system is that it takes a lot of CPU cycles to build each application from source. If you don’t mind keeping your computer on and generating heat for extended period of time, such “need” should not bother you much. If you are mobile or have a need to shut down your system form time-to-time, you may wish to plan your windows of opportunities to use Macports. This means less practical for laptop users, who typically have a need to shutdown the laptop from time-to-time.

Furthermore, Macports is only single threaded. This can be both a good thing and a bad thing for dual-core CPU owners. It can be a good thing because you will have the other core doing something else while one core is busy working for Macports. It can be a bad thing if you don’t need to use the other core and wish that Macports can take advantage of all the processing power you have. To me, this is where I wish to see Gentoo’s behavior. Back when I had only one slow core running Gentoo, the system can be 100% busy compiling sorce code while I do my regular light-duty Internet browsing and emailing without feeling any lag. I could even pushed the system a little by watching a video, which normally pushes CPU utilization higher on a slow CPU. The trick that Gentoo did was to set the priority of the activities emerge to low to avoid slowing down any user activities. If Macports can do something similar while multi-thread the compilation process, it will be a lot more efficient, especially with the new quad-core CPUs.

In my short experience with Macports so far, it has failed to install Gnucash and MySQL correctly for me. Such efforts cost me many hours of troubleshooting and searching through the Internet for possible solutions. The out-of-box experience certainly left me underwhelmed when comparing to rest of the OS or even to popular Linux distributions.

Am I bitter? Not at all! This is kinds of stuff I deal with every day and I am learning a lot going through all these hoops. If I don’t try to experiment with all these technologies, or attempt to break it, I won’t know how it works. ;) Even if I end-up not able to resolve some of the issues, I learned to appreciate with a sense of accomplishment when I am able to solve some of these wicket problems or finds a much easier or efficient alternatives. Such as installing MySQL from a pre-build package in .dmg format, than trying to agonizing over the issues when using Macports or found out after the fact that Fink’s MySQL is one release behind what I need. :)

webtest and jWebUnit: first impression

I spent a little bit of time check out both webtest and jWebUnit. Here are my first impression:

webtest is declarative and XML configuration file driven. If you like configuration management more than programming, you might like this format. All you need is edit the test definition and hit the run command, no compilation necessary. If you have well formed HTML and working JavaScript, then you should not have too much trouble running this. My initial simple test case works once I commented out the non-working and irrelevant JavaScript reference in the HTML. I tried to take another mini-step forward by attempting to “configure” a click to select one of the link items in a JavaScript widget. Webtest has 3 ways to click on a link, by ID, label, or xPath. The widget’s link item doesn’t have an ID, so I can’t use that. I tried label and that didn’t work. Then I tried xPath and that didn’t work for me either. So, that second mini-step didn’t go very far for me. Any one know an open source xPath validation tool or web site?

jWebUnit is one of the many unit testing frameworks based on jUnit/HtmlUnit. The interesting aspect of this framework is that it supports both HtmlUnit and Selenium test cases. jWebUnit uses similar programmatic model as jUnit, so you can stay comfortable within your favorite IDE. I was able to create a new project in NetBeans and leverage it’s jUnit feature, by creating an unit test and rewrite that as a jWebUnit test case, while using NetBeans to drive the test execution. jWebUnit, or the underlying HtmlUnit engine appears more picky about JavaScript than Webtest, I had to start tweaking YUI libraries in an attempt to load a page, which shouldn’t be something that I supposed to be doing anyway. Perhaps YUI engineers use a different unit testing framework. The test page actually loads without a major issue in Firefox.

Both frameworks do share one thing in common, both do not use actual browser instances to drive the testing. And that explains why they don’t work as well as some of testing frameworks that built within or on the real browser instance because of behavior differences when processing JavaScript. At the same time, both may have one advantage over browser based testing framework, which is the ability for non-interactive testing through ant and is very useful for continuous integration servers such as CruiseControl. Again, I’ve only looked a few testing frameworks and let me know if you know a browser based unit testing framework that works with ant. Thanks.

Comments and notes on Wikinomics (part 1)

A person I know urged me to read Wikinomics by Don Tapscott and Anthony D. Williams. So, I started reading. So far I read chapter 1, 4, and 5. I agree with less cheerful reviewers of this book (here and here) that this book probably doesn’t worth my time spent reading it. I actually agree the general theme presented by the book (what). I just wasn’t so thrilled by the way authors presented their ideas (how). One of the reviewers, Timothy Haugh, clearly pointed this flaw out by saying “Wikinomics suffers from a common malady among these ‘cheerleading for the future’-type books; namely, that what they take 300 pages to say could have been more easily and better said in about 60 pages.” Other than a few interesting anecdotes and catchy slogans, I didn’t feel I am learning something new and useful.

Here are a few notes on interesting anecdotes and catchy slogans:

* Low-cost of entry: Page 12: “What’s more, people can contribute to the ‘digital commons’ at very little cost to themselves, which makes collective action much more attractive.”
* Page 12: “A world where only the connected will survive.” This is digital divide and the authors says nothing about how to bring in the un-connected world.
* Globalization: Page 14: “Today 180,000 people work, live, learn, and play on Foxconn’s massive high-tech campus, designing and building consumer electronics for teenagers around the globe.” How this related to the central theme of the book? Did the globalization of FoxConn happened because of wikinomics or could it happen even without wikinomics? This reminded me the following sentence from “What Got You Here Won’t Get You There” by Marshall Goldsmith and Mark Reiter, “I’m talking about the difference between success that happens because of our behavior and the success that comes in spite of our behavior.” (Page 26)
* Page 20: “The new art and science of wikinomics is based on four powerful new ideas: openness, peering, sharing, and acting globally.”
* Page 24: “BiOS (Biological Open Source License)”
* Page 26: “But if your invention can be replicated at no cost, why should anyone pay? And if no one pays, how do you recoup your fixed-cost investment?” Where is the answer?
* Page 27: “The first time Michael Powell, then chairman of the Federal Communications Commission, used Skype, he concluded: ‘It’s over. The world will change now inevitably.'” So what? The world is changing all the time. The old world telcom industry still exists today and are getting bigger. What’s the point here?
* Page 32: Prosumer = “producer consumers” This word has many conflicting meanings according to wikipedia
* Page 104: “In fact, virtually all companies with sizable patent holdings are now busy mining their portfolios, looking for licensing-out opportunities, and taking technologies off the shelf that can bring in revenue.” What’s so wikinomic about this?
* Page 107: “When the company finds those good ideas, Huston says, it brings them inside, where it can capitalize on internal capabilities to enhance or append the offering.” Isn’t this an old concept? i.e. acquisition
* Page 107: ideagoras = “they help link all of these individuals, companies, and organizations together by establishing connections and facilitating transactions between buyers and sellers of ideas and technology.”
* Page 114: “‘Open innovation efforts can die like a flash in the pan,’ says Stern, ‘because some senior executive says, ‘I’m giving you two people and an external budget of ten thousand dollars, and we’d like to see ten deals in the first year.’ It just doesn’t happen. It takes quiet a bit of time for these initiatives to mature.” Many people probably would like to show this to their upper management this statement. However, it convey no potential value and such argument probably won’t go very far without clear communication of value-add.
* Page 114: “Employees should also be rewarded for spotting and acquiring external ideas.” I would love to see this happen. At the same time, I don’t have high hope for this.
* Page 117: “Superior technology alone will not produce competitive advantage.” Next…
* Page 129: “But when it came time to rethink the telematic features for future models(such as GPS navigation), the company released a digital design kit on its Web site to encourage interested customers to design them instead.” We should let our customers design our products too! ;)
* Page 144: “Users exert editorial control by clicking on the digg button for each story they like. Articles that receive the most diggs are promoted to the homepage. And so the community collectively updates the front page. You could say the community is the editor.” We should let our users be the editor too!
* Page 146:

* Give users access to raw content such as interviews as a 
means of providing greater transparency and accountability.
* Provide tools and become a platform for user-generated 
rather than firm-generated content.
* Redesign all content to be a conversation rather than a 
corporation monologue.
* Treat advertising as content too.
* Use new distribution forms, including peer-to-peer networks.
* Adapt content forms and schedules to user demands.

* Page 148: “If you make it profitable for customers to get involved, you will always be able to count on a dynamic and fertile ecosystem for growth and innovation.”

First impression about ColumNav 1.0

Special thanks again to David Lindquist for adding a new feature in the most recent release of ColumNav 1.0, which is the ability for ColumNav widget to load arbitrary HTML in one of its columns. This feature can really enrich the widget to go beyond a tree listing of menu items.

Yes. You can certainly display arbitrary HTML. The HTML might render a little differently than when the same content was in a HTML page. For example, if you use a <div> in your arbitrary HTML, you might get a non-scrollable scrollbar similar to the example below:
Coumnav 1.0 with arbitary HTML

So, depending on what HTML content you want to include in ColumNav and what browser you use, your mileage may very. For me, <div> is not a must have for me. If I have more time to play with this, I may try to use other HTML elements instead.

Not so dynamic “Animated Collapsible DIV”

I had a lot of fun playing with Animated Collapsible DIV. However, when I wanted to dynamically insert the content of the collapsible DIV, it would not work. I realized that it would only work for static content that doesn’t change on fly. The Collapsible DIV is initialized at page load and doesn’t dynamically change size if the content in the DIV changes. Bummer, I wished it would do more.