The Ethics of Error Prevention

Michael Feathers gave a talk at JAOO about the ethics of error prevention. He mentions that preventing errors in applications have many different solutions, but most of the time we only need to pick one or two. He then moves on and coves 5 techniques:

1. Abstraction: Use of object oriented languages and higher end languages.

2. Design by contract: document precondition and post-conditions. The clients of the routine are obligated to fulfill them.

3. Clean room engineering: discipline of annotating code that proves to yourself that the code you are writing is correct.

4. Test driven development: write a test for a new capability, compile, fix compile error, run test and see it fail, write code, run the test and see it pass, refactor as needed, repeat.

5. Pair programming/software inspections: Fagan inspection: planning, overview, preparation, meeting, rework (looping back to planning), follow-up

Michael wraps up by mentioning that each of these techniques forces us to focus on what we are doing and steers us away from complication. They each trigger contemplation. The craft of software development is not about languages or tools. It is about practices. Quality is in the intangibles.

This presentation is available on InfoQ at http://www.infoq.com/presentations/error-prevention-ethics

Refactoring Databases: Evolutionary Database Design

Pramod Sadalage gave a talk about refactoring databases. He starts by stating that business requirements change all the time and with agile practice we are getting better at embracing these changes, but that we must also get better at managing change at the database level. He then covers some recommendations to make database refactoring possible:

1. Version Control Database Assets: Having these in version control provides greater control over changes, couples databases and applications, integrate in version control instead of database.

2. Swap best practices: educate DBA about coding practices, educate developers about SQL, automate tasks (physical table deployment, usage statistics, schema verification, data migration verification)

3. Give everyone a sandbox: each developer has his own copy of the database. Have a script to create a database, create all the tables, views, etc, insert seed data and test data. Use dbdeploy tool.

4. Continues Integration: Since sql script is in version control, then it can also be part of continues integration. Migration scripts always get tested.

5. Scaffolding: The old and new schema work together until all apps have been migrated to use the new schema

6. Controlled Release: merge all deltas into release script, branch sql script with code

7. Assert Database Behavior: test the database behavior. Check for unique indexes, nulls, other constraints

This presentation is available on InfoQ at http://www.infoq.com/presentations/refactoring-databases

10 Tips for Successful Agile Transition

Joshua Kerievsky gave 10 tips for successful agile transition at QCON San Francisco 2007:

1. Begin all transitions with a readiness assessment: Figure out if you are ready and if you are what kind of agile to apply.

2. Leverage the power of non technical and technical practices

3. Evolve by learning a little of everything. Start with a whole. Do not start with a fragment. Don’t do fragile agile. Whole enchilada transition: Broad brushstroke transition

4. Educate the organization antibodies: use pilot project community and begin educating right away future project communities. Don’t sell pair programming. Sell the risks of solo programming (tunnel vision, less productivity, less knowledge transfer, longer times fixing bugs, less code re-use, poorer testing, etc.)

5. Business trumps process: most of the time, but not ALL of the time. We need to manage the technical debt.

6. Engage the entire organization: Management, customers, support, development

7. Pick a big important project: Do some little ones to start, but at some point in the transition, you have to do a big one to avoid the antibodies.

8. Handle scaling problems: use parallel knowledge transfer for training part using e-learning

9. Metrics: Do some on your own, put if you want to get serious read Larry Putman’s book

10. Fail fast: use automated tests. Work iteratively.

This presentation is available at InfoQ at http://www.infoq.com/presentations/10-tips-for-agile-transitions

Craftsmanship and Ethics

Bob Martin gave a talk about craftsmanship and ethics and wondered if we as developers have a profession. He defined a profession where as a body we can agree on a set of rules to follow. With the rise of agile techniques, developers have become more disciplined. Bob covers the following disciplines:

a. Short iterations (1 or 2 weeks). Plan, code, testing, documentation (complete cycle resulting in deployable software.

b. Don’t wait for definition: instead participate in the definition process by demonstrating working code.

c. Abstract away volatility: look at what is likely to change and separate it from what is not likely to change. (do not mix gui with business rules).

d. Commission instead of omission. It is better to experiment than to wait.

e. Decouple from others: write stubs to ensure your code can run and to define your interface

f. Never be blocked: always find some way to make progress

g. Avoid turgid viscous architectures: they do more to impede progress than it is to help. Built to create perfect architecture to solve all problems that may come up. Go with simple architectures.

h. Incremental improvement: check in the code better than when we checked it out.

i. No grand redesigns: they generally do not work very well. “tiger” team constantly trying to catch up with maintenance team

j. Progressive widening: Add a small feature from top to bottom (GUI to database),

k. Progressive deepening: Get something working in one layer and then move it down to other layers. 1st make it work, then make it right, then make it fast.

l. Don’t write bad code: It does not only slow down others months from now but will slow you down immediately.

m. Clean code: Only way to go fast is to slow down a minute and write clean code. Readable, obvious, small functions, good names

n. TDD: Follow 3 rules: 1) not allowed to write production code until you have a failing unit test, 2) you are not allowed to write more of unit test than is sufficient to fail (not compile is failing), 3) you are not allowed to write more production code than is sufficient to pass. This forces us to keep the code executing all the time. Tests make the software flexible and maintainable.

o. QA should find nothing: our attitude and approach should be that QA should find nothing.

p. 100% code coverage: strive for constantly achieving 100% code coverage. Managers should not use this number. Professionals should use it.

q. Avoid debugging: Don’t jump into the debugger. Look at the code 1st. Use TDD. Debugger should be last resort.

r. Manual test scripts are immoral: Playing with system should be explorative testing. The other vast majority of testing should be automated.

s. Definition of Done: All the tests are passing.

t. Test through the right interface: Do not test business rules through the gui. Test gui connected to a dummy (mock) layer.

u. Apprenticeship: We have to grow the new members of the team and transfer knowledge to them

v. Use good tools: open source community care about the code they write, like svn

w. Green band on wrist: A symbol of ethics. “I will be a professional developer”

This presentation is available on InfoQ at http://www.infoq.com/presentations/craftmanship-ethics

Perfection is an Unrealistic Goal

At QCON 2007, Linda Rising gave a talk about the best way to work as individuals as opposed to a team or team of teams. She mentions 2 myths:

1. We can understand it enough to get it right – In reality, we will not get it right, but we might get good enough. We are overly optimistic and believe we are better than we are. We deceive ourselves about what we want, why we want it, and whether or not we are capable of getting it.

2. The process to reach it is linear. We are hardwired for cycles (not circles). We are constantly testing the waters, taking time for reflection, celebrating success, step by step and repeat cycle. This is similar to the agile way. You can’t plan it all from the beginning. You see the goal and the next step more clearly after each step. You learn about the goal and adjust the goal. So do the customer/stakeholders. Change continues to impact the journey. Perfection is never reached.

Next Linda discusses sleep cycles. We sleep in 90 minute cycles. Each sleep cycle is composed of:

1. Light sleep, non-rapid eye movement (NREM), muscle relaxation, lower body temperature, slowed heart rate.

2. Completely asleep: NREM, further drop in body temperature and relaxation of the muscles. The immune system repairs damage.

3. Deeper sleep: NREM, metabolic levels are extremely low.

4. Delta or REM sleep: eyes move back and forth, blood pressure rises, heart beat speeds up, brain activity increases, and sleeper becomes paralyzed. Most restorative part of sleep. Most dreaming occurs.

Then Linda discuss day time cycles. In day time, we move between expenditure of energy and renewal of energy. We should manage our energy and not our time. We are under the illusion that being more productive means either multi tasking or working longer hours and not taking breaks. If you are in IT, your mental energy is critical (capacity for focus). If you switch from a primary task (programming) to a secondary task (email), the time it takes to complete the program increases by an average of 25%. Imagine impact when we check emails 50,75, 100 times a day. A study on pair programming showed that 90 minutes is the optimum duration.

Linda mentions that neuroscientists no longer believe that we peak early and then begin to die (starting in 30s). We can now build our brains continuously. It’s a never ending job. It is affected by the way we live. To expand your left brain, take a break, play, do something different, read new kinds of articles and books, visit new places with a new agenda, do these kinds of things often.

Linda wraps up by stating that perfection is an unrealistic goal. Improvement is more realistic. Not 15% by the end of the quarter but by 1% by the next iteration. She recommends we find our own cycle, focus without interruption for about 90 minutes and then taking a break for 15 to 20 minutes and repeat.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Perfection-Is-Unrealistic-Linda-Rising

Embrace Uncertainty

At CommuniTech, Jeff Patton gave a talk about agile software development. Jeff starts by giving an introduction to agile. He mentions that it is nothing new. Scrum (1986), Crystal (1997), FDD(1998), XP(2000) all lead to the agile manifesto was in 2001. Agile development describes a class of approaches and not just a single approach. It combines elements of Scrum, DSDM, Crystal, FDD and XP.

Jeff then gives a primer on Scrum. A product owner sets product goals and adds requirements or features to a product backlog, he then takes highest priority items and elaborate them into details and start a development cycle of 2 to 4 weeks. Team has a daily stand up meeting to follow progress. At the end of the iteration, they evaluate the working software and get feedback which might affect the product backlog and the next iteration.

Jeff shows how the picture of the scrum process looks like a snowman. The head is stories that get packaged into an iteration. Every iteration does not have to result in a releasable product. It’s a number of iterations that form a release, and a number of releases form the product. Each cycle feeds into planning. At the iteration level, feedback is used to better understand the problem and to reduce risk. At the release level, the feedback ensures proper planning for releasing value. At the product level, the feedback is to update the roadmap.

Next Jeff presents 3 very entertaining scenarios (using personas and music lyrics) which demonstrate some practices that might lead to problems when using an agile model. He concludes that iterate does not mean increment. In incrementing we are piecing things together and it calls for an early well formed idea. Iterate builds a rough version, validates it, and then slowly builds up quality. Quality here refers to look and feel, characteristics and features. The client needs to prioritize the goals that generate return on investment. The developer needs to know what the client wants but always keep in mind what the ultimate goal is. They also need to push decisions to the last responsible moment and build up feature quality iteration by iteration. Instead of writing stories about what to do, write stories about what it intends to accomplish. Have a goal, take action, evaluate the action, and then evaluate if the goal got accomplished. Push decisions to the last responsible moment. Each story has the following characteristics:

1. Necessity: The minimum needed to get working software.

2. Flexibility: what are some alternative ways of doing it, what additional data we want to capture.

3. Safety: better validation rule to avoid ugly error messages.

4. Comfort, luxury, and performance: more usable, sexier to look at (animation), hot keys.

Jeff recommends starting out with the necessities and then slowly build up the product.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Uncertainty-Jeff-Patton

Introduction to Project Estimation

At Devoxx08, Giovanni Asproni gave a talk on Project Estimation. He emphasized the need to distinguish estimate from targets and commitments. Estimates are approximate estimation of the value, number or quantity of something (2 to 4 days, 10 to 20 million). A target is a desirable business objective (Support x users, do not exceed 1 million dollars). Commitment is a promise to deliver functionality with a certain quality by a certain date (search functionality will be available in next release). These three are independent of each other, but you should set targets and commitments based on your estimates. The estimates are not negotiable. But the targets and commitments are. The estimates are used to determine if the targets and commitments are realistic. Usual estimates include size, effort, time, cost, risk. Given some constraints (time and cost), estimate the other.

Estimates should be accurate and not precise. This means use the correct measurement unit. If something will take years, do not estimate in days or hours. Hours will be more precise, but it will be less accurate than a more appropriate measurement unit like years.

The cone of uncertainty shows that your estimates are off by a margin of error at the beginning and gets more and more accurate as the project moves along. The cone is the same for sequential or iterative projects. Do not commit at the beginning of the cone, but at a later date after several iterations or at the product design specification.

People who do the work should create the estimates. All assumptions should be taken into account. All tasks should be included. Make sure to plan for leave/absence and side tasks such as phone, email, meeting, browsing.

There are several techniques for estimation:

1) Count, compute, judge: count if you can, compute if you can’t, judge as a last resort.

2) Mathematical (COCOMO): Tools available but based on “knobs” (inputs unknown/inaccurate).

3) Historical: Industry, company or project data. Can be accurate, avoids wishful thinking.

4) Analogy: Compare to similar projects, simple to implements but subjective and less accurate.

5) Proxy: Use proxy to effort like story points or t-shirt size. Can be highly accurate but require experience.

6) Decompose and recompose: Split into small chunks and aggregate. Can be accurate but be careful because sum of individual chucks will not equal whole.

7) Expert Judgment: Individual or group (Delphi, planning poker). Can be highly accurate, but need experts.

More than one can be used at the same time as a sanity check. Start with historical data if available. If not use, expert judgment, proxy, or analogy. Keep track of your data and use it to improve as you go along. Estimation is an ongoing activity. Refine the estimates as you go along and remember that estimates will always be wrong!

This presentation is available on Parleys at http://parleys.com/#st=5&sl=1&id=14