Revisions

Confidentiality and Process

This document is the property of Business Transformation Services.

In the interests of a successful and timely completion of Year 2000 projects, the document is made available for general use.

This document may thus be freely copied and used subject to the following provisions:

• If any part of the contents of this document are incorporated into internal documents of any nature, proper recognition should be given to the author and Business Transformation Services.
• If the document is used in its entirety, it should be distributed in its current form.

Disclaimer: The author, nor Business Transformation Services, can accept no liability for any unsuccessful results that may flow from the use of this document and any aspect of it.

Every care has been taken to ensure the correctness of the information provided in this document, however, considering the urgency of dealing with Y2000 problems, expediency has taken precedence over rigour.

References

Contents

This document describes a generic approach to managing Y2000 projects. But why have another Y2000 methodology?

The Year 2000 Software Problem presents the IT industry with some never-before faced challenges.

The word deadline comes from a line that was around prisons of old. If a prisoner crossed over that line, he was shot. No warning given.

Now the IT industry is faced with a project that has such a deadline. What’s more, almost every organisation world-wide is faced with it, at the same time.

In fact, project management is just not adequate to deal with the challenge. Senior IT project managers will have to learn, very quickly, to become programme managers.

The date is fixed, resources limited, technology will vary in every organisation resulting in unusually high risk.

What every project manager needs, like a hole in the head, is to learn a new method.

What we are presenting here, is the application of familiar techniques, to a complex problem.

This methodology distinguishes itself from its peers in the following ways:

• Risk management is fully integrated into the approach.
• A programme management approach.
• Non-"millions-of lines-of-code" focus.
• Application of IT industry best practices.
• Reference to and incorporation of the latest ISO/IEC and IEEE standards.

Further, we proceed from the following two basic assumptions:

1. It will not be possible to find and fix every Y2000 problem before the deadline.
2. Even if we were able to do so, we may not want to fix everything, for cost and other reasons. For example, if a secretary is using non-compliant technology for the purposes of creating documents, and dates can simply be typed, there may be no compelling reason to do an upgrade; especially if the money and time could be put to better use on more compelling problems.

In essence, Y2000 projects are no different from many other familiar type of project.

So why make such a fuss of Y2000 projects? Mainly, because there are some characteristics of Y2000 projects that demand a slightly different approach.

Usually, when speaking of critical success factors (CSF), we would think of factors like time, cost, scope, quality and communication. They all apply here, but there is tendency towards a particular alignment of CSFs.

Firstly, there is the fixed end date. In the case of Y2000 projects, it cannot be moved. No legislation, no edict from the pope, nothing, will change when 1 January 2000 will occur.

Secondly, there is the matter of quality. South African IT staff have tended to lag in the ability to find most bugs before handing applications over to their users. In the case of Y2000 projects, this is not acceptable - so, much more attention will have to be paid to the quality aspects - reviews, walk-throughs, testing, and so on.

Thirdly, cost. One may take the high-handed approach, which says, it will cost what it will cost, however, this is not likely to win lifelong friends - and remember, we will probably need jobs after the Y2000 has finally been put to rest.

What is a given, is that costs will rise, as the gap between supply and demand gets greater. Better, would be if we could show our customers that we are doing everything in our power to keep those, inevitably high costs to a minimum.

Finally, there is the matter of communication. There is nothing worse than nasty surprises. Y2000 projects offer the opportunity for many.

So far, so good, as to what are considered to be project CSF’s. Here however, is another, one you will agree, that becomes almost paramount: risk management.

If we accept the conventional wisdom that 50% of organisations will not be ready in time, what do we do? The trick will be in choosing which items to focus on, so that the organisation is not unduly threatened.

Programme management ensures that there is a common vision across all sub-projects. Furthermore, it ensures that the failure of one or more projects will not jeopardise the success of the programme as a whole. It considers that the whole is made up of many independent and interdependent chunks that are tangible, achievable and measurable. It ensures that the interfaces (between projects, partners, applications, etc.) are managed effectively, especially where dependencies occur.

Based on the flock of geese principle, it ensures that the end objective is not dependent on one critical component. It is concerned with business risk, as opposed to project risk, so that the project objectives may be modified if these are found to not be in line with business objectives. Finally, it recognises that there are many different strategies that may be applied, given the characteristics of a project.

In keeping with the programme management approach, the tendency to focus on one deliverable (or set of deliverables) to the detriment of business objectives is avoided and each deliverable is merely seen as one part of the overall goal.

The accompanying diagram describes the different components of a Year 2000 Programme.

Some of the key functions of the programme office would be to

• Co-ordinate and manage all activities across all business units.
• Ensure that all IT and non-IT impact areas are addressed.
• Reduce learning curve by sharing processes and techniques amongst partners.
• Track overall business risk and ensure that resources are applied in accordance with risk-defined priorities.
• Ensure clear and complete communication across all partner groups.
• Co-ordinate partner communication with vendors.
• Audit progress and compliance of impacted areas.

During 1995, the US Dept of Defence contracted a group of highly respected industry experts (called the Airlie Software Council) to define what would be considered to be the best practices in the IT software development industry.

9 best practices were identified. We believe that every one of the "best practices" is applicable in a Y2000 project, and we have structured the methodology to take them into account. In addition, we believe there are some additional best practices that extend the work done, that are specific to Y2000 project.

Adopt a programme management architecture (there are just too many interdependent projects to manage independently).

Apply robust, Y2000-specific project management life cycles that take into account the different compliance approaches and strategies.

Use appropriate tools (researchers have shown that the starting time for a large Y2000 programme aiming for completion by end 1999, without effective tools, has passed).

Validate conversion strategies and approaches against the plan. Whilst certain conversion strategies and approaches may be the most desirable, the plan may not allow for such "luxuries". For example, there has been much debate that compliance approaches like windowing or language extensions are merely postponing the problem. However, from a business risk perspective, this may be the only workable approach. This validation needs to be done at each project review point. If this approach becomes a paradigm, then projects will be planned with the assumption that it may need to be necessary to change course mid-stream.

Recognise that there are probably as many opportunities as there may be risks. Plan flexibly, so that opportunities may be capitalised, when they are identified.

Part of the programme management approach would be to identify additional "best practices" and "worst practices" and communicate these to all partners.

In addition to these "best practices", the Airlie Software Council also identified 9 "worst practices". We believe that these are as applicable to Y2000 projects.

Caveat Emptor!

The Zachman Framework for Enterprise Architecture is a model of the real world as it applies to an enterprise. Models are abstractions of real objects (which Zachman calls "artefacts") or situations (e.g. a risk occurrence). In fact, Zachman suggests that it would be possible to apply the framework to a much broader scope, namely, life itself. That, however, is not the subject of this exercise, except to indicate, that if it can be applied to something much larger, it can also be applied to a more constrained example (like an enterprise).

Models are useful because they allow us to take a very complex real-life and abstract it down to a level where we can deal with it in a manageable fashion. For example, consider the problem of two trains hurtling towards each other on the same track. If the problem is "what will the impact be when they collide", we probably do not need to consider the context (involving the weather at the time, whether the wheels were made of stainless steel, what the driver’s name was, who was the sultan of Brunei at the time, or whether Bill Gates was still the richest man in the world), in fact, we can probably abstract the problem down to a simple mathematical formula. One, in fact, which could probably be resolved by a high school pupil. It obviously depends on what the question is which will determine which factors must be included in the abstraction (or model). For example, if Bill Gates is on one of the trains, that fact may be relevant, or it may still not be.

Since humans (and computers even less) are not very good at dealing with masses of facts, it is imperative that we break the problem down (abstract it) to a level where it can be dealt with. It is amazing how simple some abstractions can be, regardless of the complexity of the problem. All will agree that there are probably only a handful of people who really understand what the theory of relativity is all about, it can be abstracted to a level where even a child can at least memorise one of its major assertions: E = mc2. His first paper on the subject was published at the age of 26 (in 1905), but the concept was already there, at age 12, when he "decided to devote himself to solving the riddle of the ‘huge world’". It was only on 29 May 1919 when his theory was verified by a scientific expedition to photograph a solar eclipse, and "the modern world began" (as the historian Paul Johnson asserts).

From observation, it is possible to organise Einstein’s work into a framework - from the concept (and even the concept can be abstracted into: Why? What? How? When? Where?) to a physical reality: the atom bomb.

The framework was first proposed in the IBM Systems Journal in 1987 and has, thus been around for a substantial period of time. In that time, it has been picked up by a number of people and applied to a variety of problems. Zachman originally got the idea of the framework from a combination of looking at the way a manufacturing company build products, and realising that it was not a miraculous invention how they went about it, but merely common sense. No doubt, his degree in chemistry, with its fundamental Periodic Table, must have got the juices flowing as he struggled with IBM’s Business Systems Planning methodology to which he was a contributor. This common sense he then applied to creating a model of the enterprise.

At is lowest level, the framework is made up of 6 rows and 6 columns.

The rows define a perspective of the problem or issue. Zachman described these to be the perspectives of the planner, the owner, the designer, the builder and the sub-contractor (as the one who actually does each little bit of work). The final perspective is (or should be) that of some result or reality.

Secondly, the columns describe a particular focus, namely, the "What? How? Where? Who? When? Why?" of whatever is being considered. So, R2C1, would describe what was required from the perspective of the owner. R2C2 would describe how the owner wanted it to be done. Note also, that whilst that may the owner’s primary perspective at that time, no cell is independent of the cells above, below or beside it.

Zachman went on to show that whilst the framework could be used to effectively represent the dimensions of the reality, the framework is in fact multi-dimensional and recursive.

Firstly, there may be any number of product frameworks in any enterprise (many iterations, which vary in content to the nature of the product).

Secondly, there is also a framework that describes the enterprise and how it works to produce or service the products that it provides. All these procedures, etc. are the model of the enterprise.

Thirdly, whilst there may be many enterprise frameworks (many iterations which vary in content to the nature of the enterprise), it is possible to create an abstraction of how the enterprise works. This abstraction, the information system, is really the meta-model of the enterprise.

Finally, we have a model that describes the way in which we create the enterprise information system. This model consists of similar artefacts that exist in the other three abstractions - and answer the "What? How? Where? Who? When? Why?" of the model of the model (or meta-meta-model). This is the framework that the CASE manufacturer uses to manufacture a repository which contains all the artefacts which are required to build the enterprise information system, which supports the enterprise which makes the product.

But there still appears to be one interrogative that is missing: "Which?" In fact, it should be evident that each framework describes the "Which?"

Furthermore, it is even possible to have parallel iterations of the various frameworks that describe the current situation ("is") as opposed to desired future situations ("to be").

Also notice the following: as an artefact passes from one row to the next, it undergoes a transformation. For example, a plan results in a number of design specs, which result in various types of diagrams (e.g. an electrical circuit diagram). These diagrams result in spec sheets, which ultimately result in a physical unit (e.g. a circuit board, or a transistor). This process of transformation usually also results in an explosion in detail. Whilst there may be less than 10 artefacts in row 1, there could be many millions of artefacts in row 6.

We believe that the Zachman Framework for Enterprise Architecture is particularly useful as a "control mechanism to ensure that nothing is missed. Since it is not known very widely as yet, we have taken the time to discuss it in more detail.

We suggest that the proposed standardised deliverable address levels 1 and 2 (Scope & Enterprise). At levels below that, the deliverables should be customised to the environment and objectives.

Year 2000 projects can vary dramatically according to the objectives and scope of the work to be done.

On the one hand, we may need fix a system that was developed internally. In some cases, the system may be so old that much of the original source code is lost. In other cases, the versions of the underlying platforms may have to be upgraded before the code can be fixed.

In other cases, we may need to upgrade a package to the latest version. This may be as simple as purchasing the latest version and letting the software do all the work for us, or it may be necessary to perform a cascade of upgrades (from v3 to v4 then to v5 to finally get to v6). Along the way, there may have been a change in technology, so that the platforms would need to be upgraded first, before an upgrade can be made to a particular version.

Then there may be cases where the original builders of the system no longer exist, and, if the software is important enough, we will have to replace it with something else, or worse, rebuild it ourselves.

The point is that there is such a wide variety of problems that no single detailed methodology would be able to address every variation and problem domain.

Thus, whilst we have tried to address the as many of the most common issues as possible, the methodology cannot be all things to all men.

In any event, project management is a "thinking head process" and what we are attempting to do here, is to provide a good starting point for the project manager who has been given a Year 2000 project and has been told "go and solve it".

Now he sits with a blank sheet of paper before him and wonders where he should start.

We have followed the general life cycle for Year 2000 projects as described by IEEE P2000.1.

We identify every task by a CoA ID (Chart of Accounts identifier), which is not the same as a WBS ID (work breakdown structure identifier), although it would be preferable that the CoA ID be used as a part of a WBS ID. Our recommendation would be the WBS be made up of CoA ID and Deliverable ID. If this is done, it is easy to record actual time spent (effort) and duration for the completion of each deliverable. This information is invaluable for calibrating estimating models.

The structure of the CoA ID is as follows: PATTS where
P: Phase (0 – 9)
A: Activity (A – Z)
TT: Task (01 – 99)
S: Sub-task (a – z)

Deliverables are identified according to the following convention:
TT: Deliverable type
NNN: Number

Exit Criteria describe the checkpoints that define completion of the task. There are 3 priorities: A (must be conformed to); B (should be conformed to, or by way of an alternative, if appropriate); C (is a useful check, though may not always be applicable).

We have created an MS Access 97 database that can be used to capture and store the information that is collected during the Assessment phase. The database is under continual enhancement (for example, at the time v3.0 of this document, the database was being enhanced to incorporate Change Control and Time Recording).

In addition, MS Excel 97 and MS Word 97 templates have been created to support various of the tasks defined by the methodology.

The following resources are available:

• Y2000 Compliance Issues. This document describes how different Y2000 compliance definitions can be rationalised to arrive at an internal definition.
• Y2000 Risk Level Determination. This document describes a framework for determining Y2000 risk.
• Y2000 Risk Assessment. This is an MS Excel 97 workbook that can be used to ensure consistent risk assessment. Weightings have been allocated to different risk factors to derive overall risk per system. The issues that are taken into account are the importance to the enterprise of the functions that the system under consideration supports, the use of dates and the amount of "work" required to fix the problem.
• Y2000 Inventory Database. This tool has been developed in MS Access 97 and includes more than 60 tables, 100 forms and 100 reports. It is intended as a one-stop repository for all Y2000-related information. Almost all of the deliverables included in this methodology can be captured in the database.
• Project Charter. A MSWord97 template of such a deliverable.
• Y2000 Estimating. This is an MSWord97 article that describes how estimating may be approached.
• PoM2000 Project Estimating Sheets. This is an MS Excel 97 workbook that uses an activity-based estimating model to derive an estimate based on the methodology described in this document. The author has found it accurate and useful if used in conjunction with the project control workbook.
• PoM2000 Project Control Workbook.. This is an MS Excel 97 workbook that can be used to apply the Earned-Value Reporting technique to control progress on a Y2000 project.
• Vendor Information Request. This document is an example of the letter that should be sent to vendors.
• PoM2000 System Description. This document summarises the information that needs to be collected to do a comprehensive description of the system (based on the database tool).
• PoM2000 Interface Description. This document summarises the information that is required to do a comprehensive description of the interfaces (based on the database tool).
• Y2000 Testing Proposal. This is an MSWord97 example of a testing proposal.
• Y2000 Testing Plan. This document follows on the Proposal and is an example of such a deliverable.
• Y2000 Test Case. This MSWord97 document is an example of such a deliverable.
• Presentation Materials. Various presentations given at conferences are available.
• PoM2000 Course. This is a 3-day course that describes the whole approach. It is targeted at the project manager, but other skill levels would also receive benefit.

Please see the Project Management Guide for the general discussion of how to plan projects. What follows is a list of the activities and tasks that would typically be performed in any project.

It is our experience that around 25% of total project effort be budgeted for these activities. This is much higher than what one normally expects for IT projects (usually 9% - 18% depending on the size of the project), but is borne out by observation.

A feasibility study is initiated by the receipt of a project request. Typically, the amount of effort is determined by the amount of time available to do the study. When the time is up, the results are documented, and presented to management. As a consequence, one or more projects may be defined. These are then described in the Project Definition.

Planning involves collecting summary information about the scope, so that a detailed estimate may be prepared.

If possible, an estimating tool should be used.

Once the size of work is known, and the levels of skills required, it is possible to schedule the work.

Rates are typically associated with skill level. This, timephased as per the schedule, together with additional costs (travelling, infrastructure, training, etc.) then go into the budget.

Finally, all of this is consolidated into the Project Charter, which becomes the formal contract between management and the project team.

Organising the project involves recruiting and selecting staff for the project team, defining roles, agreeing what reporting will be required, and setting up the physical environment - work areas, hardware, software, and so on. A kick-off meeting is held to describe how the project will be run, its objectives and so on.

Risk Management is one of the most important activities in any Y2000 project.

Two types of risk need to be distinguished: Business risks, which describe what will happen to the business should we fail in any way; and Project risks which describe what could cause us to not complete on time.

Understand that inherent in any risks, there are also usually opportunities. These opportunities typically involve an entirely new set of risks.

However, it may only be from taking risks, that we can achieve the best results.

Y2000 projects cannot afford defects to creep through. After all, this is the reason we are conducting Y2000 projects.

Communication within projects and between projects making up the programme is vital. At every step of the way, concerned parties must be kept informed about how the work towards readiness is progressing.

Once the Project Charter has been accepted, the estimates and schedule become the baseline against which progress and costs must be measured.

Inherent in every Y2000 project is change - scope will fluctuate as the degree of compliance is revealed. Missed items will be identified.

To retain control, every one of these changes must be recorded, course of action determined, approval obtained and then tracked in concert with the baseline plan.

Similarly, the project environment needs to be kept up to date and modifications made where necessary.

The format of deliverables should be modified as the need arises.

Project control involves regularly checking on the status of work, determining whether the plan is still on target. If not, then contingency action must take place, or the baseline must be reviewed. The plan can only be changed if management approval is first obtained.

Often changes in approach, or re-assignment of staff can make the difference to bring the project back on track again.

The best form of control is still via management by the sole of the foot!

Contract administration involves ensuring that the relationship between all parties involved in the assignment is at an optimum level. That there are no undue risks as a consequence of improperly defined responsibilities, that everyone is paid correctly and on time, and that all issues of due diligence are properly taken care of.

Experience has shown that most organisations have taken up to 9 months to pass through this phase.

Usually, the Y2000 champion is faced by disbelief that there could be such a problem (after all, what difference could two digits in a date make - be real!). This is then followed by denial (it can’t be that big. Bill Gates will invent a tool that will fix it. We spent millions to buy a system that would last us 10 years. You promised us.)

When the nature of the beast really dawns, panic often sets in.

Since we just do not have 9 months, in most cases, the Y2000 champion must convince senior management as soon as possible that there is a problem, and how it can be solved.

Some early estimating techniques to determine impact and cost have been proposed by groups like Gartner ($x per line of source code), the British Computer Society (Rx per head working in the organisation) and Capers Jones (cost per function point, normalised to industry and technology). By mid 1997, many estimates had already started appearing in the press, so that organisations can find comparable situations elsewhere as an order-of-magnitude measure.

None of these measures are likely to be definitive, and it is necessary to move to a Discovery & Assessment Phase. However, to be able to do this, one needs to obtain commitment and resources. These are the major challenges of the Awareness phase.

In practice, the Awareness exercise will probably continue in parallel with other activities, because the level of awareness varies within the enterprise, let alone in the supply chain.

We will not say much more here, except to mention that there are many resources available in places like the Internet, which include descriptions of the problem, presentations, and so on.

Some of the most useful sites on the Internet are:

By 1998, numerous books have been written on the subject. Some of which have been included in the references at the front of this document.

In addition, there are a seemingly endless list of conferences and there is hardly a week that goes by without extensive references to the problem in the media.

There are many differing views on compliance. Here are some definitions.

The IEEE P2000.1 Standard for Year 2000 Terminology has the following definitions:

The British Standards Institute views compliance this way:

Others have described less complex definitions. For example, the US State of Washington, describes Y2000 Compliance as follows:

"Information systems are Y2000 compliant if they are "able to accurately process date data - including but not limited to, calculating, comparing, and sequencing - from, into, and between the twentieth and twenty-first centuries, including leap year calculations."

The definitions assume that all software must be able to process dates correctly. Since this is unlikely to be true of many older applications, this could result in enormous costs, and more importantly, reduce the chances of being ready on time.

Organisations therefore need to take a more pragmatic view and focus on the software items that are most critical to the organisation and assess to what degree compliance is important.

For example, many fax machines have the ability to print the date and time of transmission or receipt, together with the sender name and telephone number. If your organisation is a firm of attorneys, where the automatic logging of date and time is important to establish when something was done, then there is no question that this needs to be fixed, but otherwise? Does it really need to be fixed?

To avoid delays later, you need to define your rules early.

Leap years are years that are divisible by 4, unless the year is also divisible by 100.

However, if the year is divisible by 400, then it is a leap year.

Therefore, 2000 is a leap year (divisible by 4 and by 400), but 1900 wasn’t (divisible by 100, but not 400), nor will 3000 be a leap year (ditto). 2004 will also be a leap year (divisible by 4, but not by 100).

The reason for these strange (or so it appears) rules, is that a year is not 365 or 366 days, but approximately (ha!) 365.24219878 days.

This matter is sometimes referred to as "Due Diligence" and it involves the responsibility of directors and others entrusted with the protection of stakeholders’ interests.

In essence, directors have the responsibility to ensure that the concern under their supervision is maintained as a "going concern". As such, they have the responsibility to take every reasonable action to ensure that the enterprise does not fail, or failing that, to make this known to stakeholders.

If they do no do so, they may expose themselves to legal action. If lives are lost, the legal action may involve criminal prosecution.

What we have been able to establish from discussion with various attorneys is the following:

• South African laws are largely "Y2000 compliant" in the sense that it would not be necessary to pass additional laws.
• There are substantial differences between the way that South African courts would address Y2000 claims compared to their US counterparts.

• SA courts tend to be more conservative and would tend to take the view that the "loss should lie where it falls".
• There is reluctance to add punitive damages.
• Documentation would tend to carry more weight than verbal or other evidence.

There are a number of firms of attorneys that are specialising in this field. If in doubt, an attorney should be consulted.

Vendors and other developers need to specify what the degree and level of compliance is. Be aware that a conditional reply may be of no value.

The scope of the project is confirmed by checking that all application areas that support a business area have been identified. As well as who the key decision-maker is. We need to also identify the systems, or applications, that make up the application area. In addition, identify the platforms that the systems require to operate effectively and the vendors that supplied the software.

This information would typically be obtained by conducting interviews with the key people. They also need to be asked about how important the systems are to the business - would business operations be unaffected if date problems had to cause errors. Establish to what degree the business would be affected.

All this data should be captured into an inventory, and the results published.

The next step is to conduct a Detailed Investigation.

Each of the platforms must be described in detail to identify whether the specific version or release has Y2000 implications. This would be obtained from the vendors who supply or support the platform.

Each system, or application, must also be described in full. Note however, that if a COTS package is being reviewed, the amount of detail will depend on how it is used. In some cases, it may be sufficient to establish that the COTS is Y2000 compliant.

In many cases, these systems interact with other systems, and it is important that the nature of this interaction is defined. Again, judgment should be used: the level of detail one is required to collect depends on the nature of the interaction.

Having now collected a great deal of information, it is now possible to review the priorities and the degree of Y2000 impact. It may also be necessary to revisit some aspects to gain greater clarity.

In the case of bespoke or custom developed systems, it may be necessary to do code and database analysis to identify exactly which lowest level items (line of code or data field) may have Y2000 implications.

Whilst one is performing this exercise, it may also be possible to identify redundant code, or improvements.

Having completed this detailed analysis, priorities and risks need to be re-evaluated. If it is found that risks are very high, it may be necessary to adopt a different approach - possibly fast-tracking the readiness project for one of the systems or platforms.

At this time, we would also determine, for each system, what the course of action should be - do we retire the system, modify it, upgrade it, or how else should we deal with it.

Then we need to define just how we may be certain that we were successful, by identifying what testing will be required, and the nature of that testing.

All of these changes must then be documented.

The full inventory is then reviewed during the Level 1 Compliance Verification, to ensure that nothing has been left out. Assumptions regarding risk and readiness must be revisited to ensure that they are still correct. Finally, the work required to convert non-compliant systems and platforms must be estimated, and the time that is required to complete the conversion work must be determined.

Typically, the results will be unacceptable because of timing and resource constraints, which means that one will have to review different scenarios to determine the most appropriate course of action.

Having done this, the plans need to be authorised by the client, so that work may start on the conversion activities.

Considering the great number of variables and concurrent tasks that need to take place, project and programme management will need to be effectively conducted.

Risk level is a summary of the overall risk that a Compliance Configuration Item (CCI) is seen to have. As the accompanying figure shows, Risk Level considers three basic factors:

• Is the CCI critical to the business?
• Is the CCI impacted by the Y2000 (or date processing in general) in any way?
• Is there likely to be a lot off work required to remediate the Y2000 impact?

This implies one or more of the following:

• The CCI is supporting a business function that is mission critical.
• The enterprise would incur significant financial or other loss if the CCI should fail.
• The enterprise would be unable to deliver the same level of service to its customers in 2000+ as it was able to do in the years leading up to 2000, if the CCI were to fail.
• A strategy that is to be implemented some time during or after 2000, that is key to the future growth and development of the enterprise, will be severely impacted if the CCI should fail.

JM Juran describes the process of Criticality Analysis as to "identify the ‘vital few’ features so that they will receive priority of attention and assets." He goes on to list what he calls the product features that should be considered as critical:

• Essential to human safety.
• Legislated mandates.
• Essential to salability.
• Demanding as to investment.
• Demanding as to continuity.
• Long lead time (required for input sourcing).
• Ethically sensitive areas.
• Instability.

This question addresses whether there is a date-related impact to do with the changeover from 1999 - 2000. The answer is either Yes or No.

This question is not so easy to answer, since "a lot" is a relative term. A lot of money means something quite different to a poor person as opposed to the CEO of a Fortune 500 company. Similarly, "a lot of work" seen in our context must be taken in relation to the enterprise. A company like Fluor manages multibillion projects as a matter of course. The same is not true of most companies. We recommend that a questionnaire be created that would place the following issues into context in the company concerned, to ensure that subjectivity is reduced and classification can be consistent.

We describe "a lot of work" in one or more of the following ways:

• The amount of effort required to fix the problem will be considerable (can be expressed in person-hour ranges depending on the typical project that the company manages). This obviously varies as we get closer to the earliest failure date (the latest date by when changes must be implemented to prevent failure, or to manage the failures, of a CCI). The effort would be spent in fixing, replacing, developing workarounds and any other activities to avoid or mitigate the risk of failure.

• The amount of time required to fix the problem. For example, if a CCI has been created and is being maintained by a supplier, the amount of actual effort may be low, but lead times and contract negotiation time might jeopardise the implementation of any required fixes.

• The cost of fixing the problem. Depending on the nature of, and other factors inherent in the enterprise, the cost requirement, or the cost approval process may be so long as to result in protracted negotiation before approval is obtained.

• The requirement for external input. This is also a time-related aspect. If a great deal of effort is required from, or key task completion exit criteria are dependent on external parties (to the maintenance project team), this results in inevitable delays and is notoriously difficult to manage.

• There are many dependencies on other projects. This may also be true, if there are few dependencies, but the dependencies are risky (e.g. the dependent aspects of the other project may not be ready in time). One of these dependencies may be the compliance of platforms.

All of these require, potentially, a lot of work to resolve. Note: the guidelines given assume that remediation projects will commence by latest 3^rd quarter 1998!

Note that none of the above lists are necessarily conclusive! The evaluation of the risk level criteria could be a lot broader than what we have described above.

That is why we have developed a classification scheme of CCIs to assist with the determination of risk level.

This classification scheme considers each CCI from a number of different perspectives. The exact classifications are summarised at the end of this document.

• What is it?
• To what degree are dates involved?
• What sort of risk is involved?
• What action is proposed?

This set of classes describes the extent and nature of date usage in the CCI.

Note also that certification is a sticky issue. Very few and quite possibly only foolhardy vendors would be prepared to offer a carte blanche certification for their product. This is because certification depends on so many different factors, including the way in which the CCI is installed and its interaction with other CCIs. As a consequence there has been significant backtracking on the part of many vendors in the last year with regards to the certification level of each CCI.

In any event, compliance level can only be verified if the certificate is accompanied by a description of the testing process (often a proprietary issue, or trade secret) and documented test results, which can be replicated consistently.

Compliance Strategy. We need to determine how the problem will be addressed at a high level. For example, elimination, repair, retirement, developing an interim solution, replacement, transfer, and so on.

All of these aspects come into play in finally determining the risk level of a CCI.

The process that is followed can be roughly described as follows:

1. Investigate CCIs. Part of which would be to ask the owner of the system what they feel the criticality and amount of work would be.
2. Document the results.
3. Complete the classification scheme.
4. Review the opinion of the system owner (with regard to risk level) against the results of the investigation and the classification scheme.
5. Revise the risk level.
6. Confirm the findings.
7. Prepare plans for follow-on work and a testing proposal on the basis of the risk level.
8. Seek approval for the general approach.

The classification of items is based on the following simplified model of the role of technology in the enterprise (see the accompanying figure).

In its most simple form, an enterprise is made up of business units, which are supported by application areas. Application areas are made up of one or more applications or systems.

Business units can be functional, geographical, hierarchical, or according to any other classification. For example, they may be Admin, Finance, Manufacturing. Or they may be Gauteng region, Western Cape region.

Application areas are typically groups of applications or systems that work together to support a particular function or set of functions, for example, Financial Accounting, Production Planning, Order Processing, etc.

The terms systems and applications are used interchangeably as any combination of software and procedures that are applied to supporting a business function (for example, Debtors, General Ledger, Salaries, Master Production Scheduling, etc.)

A platform is any technology that is required by the application or system

• in the normal course of its operation (i.e. operating system, database management system, utilities, hardware, etc.);
• to maintain it (e.g. programming languages, CASE tools, etc.); or
• in its original creation (e.g. programming language).

Platforms therefore only have a reason for existence by virtue of the applications that require them.

For example, if the renovation strategy is that a system will be replaced with new development, then the sub-project would be made up of the tasks that are typically associated with a new development framework. If the strategy is to repair, the sub-project would be made up of all the tasks typically involved with a maintenance framework. Instead of a project per system, one could group similar systems, or the systems that make up an application area. Alternatively, systems could be group by strategy.

However one groups remediation work, risk must be considered. It would generally be bad practice to group together high-risk systems with those of lower risk.

In fact, if a system is high risk (Risk Level is A or B), one should consider managing it as a separate project. This ensures that the focus on high risk systems is not lost.

When lower risk systems are grouped with a high-risk system, there is the possibility that attention will focus on the lower risk items to the detriment of the high-risk system.

The first step in remediation is to review the strategy that was proposed during the Assessment phase. One needs to confirm whether this strategy is still the right one – especially if there has been a number of parallel Assessment phases (typical in large enterprises).

The activity is concluded with the acceptance of Project Charter(s) for the Remediation Phase. A separate Charter should be created for each remediation project.

Having obtained management approval on the way to go, the next step is to prepare for the renovation work. If platforms are non-compliant, then they need to be updated (typically involves a "patch" or a "service release) or upgraded (usually involves a new version). The amount of work varies quite dramatically, depending on whether we are dealing with an update or an upgrade. Updates are often free, whilst upgrades must usually be paid for.

Upgrades may also involve changes in functionality. Therefore, it is possible that follow-on work may be involved (for example, if functionality has been added, or existing functionality has changed, or underlying data structures have changed).

Then, if a platform runs on other platforms, the upgrade may result in a cascade of upgrades. For example, the enterprise uses an MS-DOS-based accounting package that is not compliant. As it turns out, the only compliant version is the latest version, which requires Windows 95. In this case, to upgrade the package, the enterprise will probably have to replace the hardware, then upgrade to Windows 95 before the accounting package can be upgraded. As an additional knock-on effect, it is possible that other software would also need to be upgraded (like the word processor, spreadsheet and so on). Then, the users will have to be trained to work with a mouse, graphical user interface and so on.

In addition, if physical renovation work has to take place, it may be necessary to prepare the technical environment for this – latest versions of platforms must be ordered and installed, etc.

If code renovation is to be done off-site, authorisation must be obtained to do so, etc. In addition, it may be necessary to obtain software factory quotes, negotiate impact analysis and / or code renovation contracts, and so on.

Having done the preparatory work, it is then necessary to design the changes that will be made. This will depend on the nature of the remediation strategy. If a system is to be replaced, the existing system must be taken out (retired), this may involve modifications to partner systems, since it is unlikely that the functionality of the replacing system will match the existing system exactly.

If a system that is no longer required, is being replaced, one needs to be certain that there are no partner systems that depend on functionality that will no longer be available.

Therefore, the design of the solution can be quite complex and an essential task.

If there are any interfaces, the changes need to be planned, designed and implemented very carefully. An interface always has at least three essential components.

• Firstly, there is the system that provides the data.
• Secondly, there is the system that receives the data.
• Thirdly, there is the actual data that is being passed.

The "owner" of the interface is the system that has control over the layout of the data in the interface, how often it occurs, what the format must be, and so on. The owner is usually the system that required the interface to be created in the first place.

One of the main challenges is co-ordinating that all of these things occur in a properly synchronised fashion.

If the remediation approach is to expand dates, then data file layouts will change. Again, there may be other systems (partner systems) that share the same data. Changes to data layouts may impact on these partner systems.

Regardless of the strategy and approach that is adopted, the changes to the system need to be designed.

If a masking approach is adopted, bridging routines must be developed.

Before the renovation is attempted, the production system must be backed up and a freeze zone started. By definition, no changes may be made to the production system during the freeze zone (other than what one would call emergency changes - statutory changes or bug fixes).

The remediation strategy is then implemented, and regression testing is performed (note: it is vital that the remediation work be limited to Y2000-related changes to ensure that the duration of the frozen zone and complexity be kept to a minimum). Regression testing merely tests that the changes that have been made have not caused the software to regress – or go backwards, i.e. the system still does exactly what it did before the changes were made.

There is often a temptation to fix other problems that are uncovered in the process. Care should be taken that this extra work does not jeopardise the successful and timely completion of the Y2000 project. Note that allowing other changes to be made would impact on regression testing, and could require changes to be made to standard test cases.

Before the system can then be transferred back into production, a delta analysis must be performed to identify any changes that were made to the production system that slipped past change control. This is typically done by doing a binary comparison between the copy made of the production system when it was transferred to "maintenance" and the production system. If any changes did slip through, plus if there were "emergency" changes recorded by change control, these must first be retrofitted to the renovated system before it can be transferred back to production.

Finally, there would typically be a "warranty period" during which time the renovation project team would be on hand to fix any failures that might occur, before the renovated system is signed off.

It is unlikely that any sizeable enterprise will have only one system that is impacted by the Y2000 problem and must consequently be fixed. Since it is quite likely that there are multiple systems, these systems must be tested specifically for the Y2000 as well as that one will not upset the other.

There is a strong case to be made that Y2000 testing will be the greatest challenge any Y2000 programme will face – not only in terms of effort and duration (there is much evidence to indicate as much as 50 –70% of total project duration and more than 50% of effort (person hours)), but also in complexity.

During Verification & Validation, we must verify that the Y2000 changes were correctly done and validate that the different system will work together correctly during the year 2000 and after.

For example, we will need to set the system date forward to 1 January 2000, 29 February 2000, etc.

Year 2000 testing is one of the most critical activities that any Year 2000 project will engage in. Practice has shown that most companies are rather poor at testing. The average system will be delivered with only around 95% of "bugs" removed.

The accompanying table shows the generic types of testing that are usually performed. Of interest, is also the percentage of organisations that actually perform those types of tests. Not all of the tests apply to Y2000 projects, and the type of test often depends on the nature of the system.

In the case of Year 2000 projects, this means that these bugs will only be discovered after the 31 December 1999, with the consequent disruptions to business and associated problems.

When modifying code, most organisation will inject one new defect for every 14 they fix (the best organisations achieve 1:50 and the worst 1:5).

It is therefore critical that proper attention be paid to the planning and management of testing. This is especially true because testing is the last major activity to occur and time will in all likelihood have run out – i.e. there will be no time for recovery from lost opportunities or unavoidable error.

What makes Y2000 testing particularly difficult and complex is the number of systems that must be tested at the same time (this may be by choice or by requirement) and the divergent nature of the systems being tested. In addition, it is likely that the testing phase will be subjected to severe time constraints.

We will describe the testing-related activities that should be executed during each of the phases of a year 2000 project in this section.

There are no specific testing related activities envisaged during the Awareness Phase, other than to warn the enterprise of the criticality of testing and the input that will be required on the part of business.

Since management is requested to approve a strategy for Y2000 compliance remediation at the end of the Assessment phase, it is important that the risks are clearly defined for review.

Associated with risk, testing must be performed to "prove" that the enterprise will not be subjected to excessive business risk (beyond what it has normally accepted and lived with in the past). Since there are broader issues involved than just the correct operation of systems (for example, there is quite possibly an issue of "due diligence" that affects directors and officers of the enterprise, and there is the likelihood that auditors will be required to report on the effectiveness of Y2000 remediation exercises), it is important that management be made aware, and have the power to influence, the plans to prove the remediation work.

It is thus important that a Test Proposal and High Level Test Plan be completed before the conclusion of the Assessment Phase, and that these documents be included as a part of the Assessment Phase exit criteria.

The purpose of the Test Proposal would be to describe to management what the different options for testing are (from no testing at all, to in depth testing), including the rationale for each. A recommendation is made.

The team that has done the assessment of systems should have a good understanding of what testing will be required. For the recommended approach (a la the proposal), a broad strategy is described – what should be done, what input will be required from the business, when it should be done, what the test bed requirements are likely to be, etc.

During remediation, whilst the changes are being made to the system, is the time when detailed test cases should be developed.

Ideally, a separate team should be working on this. They can act as check on the remediation team to ensure that the most important objectives are being achieved. If the remediation team develop the test cases, there is a chance that the time pressure will seduce them into cutting testing, or developing test cases that prove what they have done (rather than proving what they should have done).

The remediation team should be responsible for unit testing of changes, limited integration testing (for example, at the function level) and regression testing (proving that the changes have not changed existing functionality, or introduced new errors.

Test beds should be readied during this phase.

The validation phase should be restricted to Y2000 testing.

The test beds should be ready, configuration management in place to manage the relationship of systems in production and test, etc.

The testing of each system should be concluded with an acceptance test, where users take back control of the fixed systems.

Implementation should not include any testing activities.

Base assumption: it will not be possible / may not be necessary to test everything. Test Proposal is created during Assessment Phase to propose to business what, why, how, where, when testing should be done, and by whom. Dec’97 Cap-Gemini survey indicated that all surveyed companies would perform partial testing only.

Purpose: to show the decision-making process involved in determining what should be tested, and what not. Reason: for the sake of "due diligence".

Test Strategy applies to those systems that will be considered for testing. (Note this important distinction!)

Test Proposal should describe how different systems may be grouped for portfolio testing (note that a system may be a part of more than one portfolio) as well as the objectives of enterprise testing.

Proposal assumes a risk driven approach. Generic strategies are:

The Testing Proposal should include the following components:

• Inventory of systems to be tested
• Inventory of systems excluded
• Rationale for exclusion of systems from scope

A high level Testing Strategy, which would include:

• General approach to be used
• Test Bed requirements

For those systems that will be subjected to testing, describe what, why, how, where, when this testing will be done. I.e. the test strategy would be a refinement of the Test Proposal, for the subset of systems that will be included for testing.

A Test Strategy is typically determined for each system, or portfolio of systems to be tested. Partition on the basis of what can conveniently be handled by one team. Note that the remediation team would typically be involved in the testing effort. This should make scheduling interesting.

Testing should generally be restricted to date-related testing. There is unlikely to be enough time and resources to do more.

The Test Strategy should address the following aspects:

• Assessment of the type of system and system elements.
• Description of the major dependencies. For example, "hub" software should be tested before systems around the hub are tested. Platforms should typically be certified before testing of systems that run on them are tested, to ensure that no unnecessary complications are introduced. Systems that have time dependencies should be noted.
• Determination of the factors critical to the validation that testing successfully completed.
• Determination of the testing scope, timing and resources (people, test bed, etc).
• Evaluation of test effort tradeoffs (risk, schedule, scope, quality, techniques, resources, cost).

The Test Strategy is typically made up of the following components:

• Test Plan
• List of Functions
• Refined Test Bed Requirements
• Acceptance Criteria

The Test Plan describes the scope, approach, resources and schedule of intended testing activities, as well as the testing environment. The Test Plan describes the specific functions, transactions and features to be tested, who will do the testing, how they will do this and when, and what contingency action will be taken should major risks be realised.

The nature and comprehensiveness of testing should be tailored to the criticality of the item. (See the Classification Codes for more information on determining criticality, impact and approach.)

It is often useful to make use of a network-diagramming tool (such as may be found in a project management tool like MS Project) to describe the dependencies and flow of a major testing activity.

An Inventory of relevant (date-impacted) Business and Technical Events should be created during the Assessment Phase. Events result in processes that are supported by Business and Technical Functions. Inventories of these, together with Function Summaries for relevant functions should also be created.

Business and Technical Functions should be categorised according to the degree of date impact and the criticality to the enterprise or business unit.

Critical Functions should be reviewed to determine the nature and level of testing that should be performed. Test cases would be devised for these functions.

Functions are performed by executing transactions. Thus, test cases should describe the event that causes a transaction to be executed, followed by a detailed transaction flow until the condition arises that causes activity to stop.

For each relevant function, identify whether (or not) the function will be included for testing and the rationale behind the decision.

There are generally two types of criteria: generic (which apply across the board, and should be considered for test cases in all test plans) and specific (which apply to a specific application).

Examples of generic criteria include testing for century change, leap year, etc.

Examples of specific criteria include for specific action that would relate to specific application-determined dates, century-spanning, etc.

As the nature and comprehensiveness of the testing is clarified, it is possible to be more specific about the exact requirements of the test bed.

Information required for the provision of the test bed includes:

• Hardware (type, capacity, configuration, throughput, level of availability required, location)
• Instrumentation tools (if available, required facilitate the validation of tests)
• Simulators (if available, used to simulate a production environment, setting dates forward, ageing data, etc.)
• Other software (configuration management tools that will facilitate the management of data sets and test cases, etc.)
• Data sources (what, how much, where from, etc.)

A Test Case specifies the set of test data, together with the associated procedures developed with a particular objective in mind. Test data includes a populated database, input values and expected results. The amount of data will depend on the nature of the testing (whether an aspect of performance is included, possible condition coverage, etc). Test cases must be defined for both internal interactions of a system, as well as interaction with other systems (interfaces).

In some instances, Pass/Fail criteria may be defined. For example, if a range is acceptable, the low and high values need to be specified.

The objective of the test case should be clearly defined (why this test case should be executed).

Test conditions should be described, which define different scenarios that would apply (for example, century rollover, century spanning). In addition, any constraints should also be defined (time constraints, dependencies, etc).

The test procedures should list the exact steps that need to be followed to achieve the desired level of certification.

Expected test results should be identified that can be used to verify a successful test.

A test log should be used during the course of applying test cases to ensure that every test step is executed according to plan.

Small incidents that are resolved immediately by the testing team, and do not require additional follow-up should be recorded on the test log.

A test incident report should be completed for every incident that occurs that requires additional action. Small issues that are resolved immediately can be included in the test log.

Considering the comment made in the introduction about the risks associated with testing (that there will in all likelihood be no time for error), it would be useful to consider some ways in which the elapsed time could be reduced.

Remember that there are usually benefits associated with risk taking (else why would you take the risk?). The key is to ensure that one receives the benefit, and is not punished (by realising the risk).

• Attempt to do as much as possible in parallel. Consider sequential processes in the testing plan. Is the sequence absolutely required? Can one, with a little extra effort, create separate test beds to test in parallel (i.e. create input data without waiting for the predecessor to produce the required output). Note however, that this will increase cost, also that the amount of checking (of output) will have to be increased and be more careful.
• Spend time during remediation to set up the test bed, especially the data sets.
• Upgrade hardware to reduce test case turnaround time.
• Is it possible to reduce the test case process time? For example, by accepting risks for the benefit of short testing. One would obviously need to consider the criticality of the function.
• Introduce formal inspection and review processes into the remediation process. Many of the best organisations have been able to find and remove more than 95% of defects through design and code inspections. The less defects there are, the quicker testing will be.
• Reduce project size. The highest productivity is typically achieved by teams of less than ten people in projects of three to six months. Try to cut the suit to fit this cloth.
• Test case design and test database design can start as soon as the following has been established: function Y2000 impact and criticality.
• Make use of tools as much as possible. Tools may be available to assist with the following:
• Version control
• Scripting
• Test bed management
• Delta analysis
• Change control
• Issues management
• Simulation

What we will need to focus on, is to prepare the enterprise for the year 2000. For example, making staff fully aware of contingency plans and fallback procedures. If the decision has been made to place less important functions on hold during the fist weeks of January 2000, and use the staff to support critical functions, then they would need to be trained.

Time need to be taken to do "what if" scenarios, to ensure that as many critical failure issues are covered as possible.

Finally, it is conceivable that processes and procedures will have changed during Y2000 Validation. These need to be implemented.