Lyle Eesley (LE): Ladies and gentlemen, I am Lyle Eesley, a professor with Defense Acquisition University here at Fort Belvoir. We will talk about services acquisition and a tool we developed at DAU. We call it the ARRT tool, the Acquisition Requirements Roadmap Tool. I want to talk about why we developed the tool and put the process in context. A little bit about me. I am with the Department of Defense at DAU, I spent 29 years in the air force, been everything from a contracting officer to section chief to a division chief and director of contacting for three major commands. Building business strategies and contracts for services- I know the importance of it. One of the things I would like you to do as you participate in this -- at the bottom of the screen is a survey box. If you have questions on anything I am saying today or would like to find out more information that I did not cover, please put a question in and we will try to answer those questions with as much time as we can to work the questions in. Please do that. I look forward to your questions.
So why is services important? Ok – what I am going to be talking about is information that I know within the Department of Defense. But I would submit that with every dollar out there it consumes a significant amount of your budget. If you look at this chart, we have been spending more on services at DoD in the past 10 years than we have on equipment and supplies. It is a huge opportunity, and we are all agencies that are facing quite a crunch in the next few years as we struggle with budget issues, deficit issues, everything else, and it will be incumbent on all of us to try to maximize the value we get out of our service acquisition dollars. So it is hugely important on all of us to do this better, and the Department of Defense, I know, has a lot of room for opportunity. I suspect your organizations do, too. Acquisition professionals, look at what your agency spends on services over a year. When you see this, the way we have constructed this, we have built what we called ”portfolios of services”, such as construction, facilities, transportation; so we can define how much money we spend in each of what we call portfolios, the taxonomy that has been created within DoD. I will talk a little bit more about that later. It is instructive to see how much money goes in services and why that is important.
Back in 2010, Dr. Carter created this “Better Buying Power” initiative. One of the things that we're focused on in Better Buying Power is to improve the tradecraft in services acquisition. We have developed this tool and the other tools we're working on to accomplish that objective. How do we improve the tradecraft, i.e., defining requirements, building better business strategies, and successfully executing and delivering goods on a service contract? How do we do that as a team? This is the taxonomy DoD has developed. Knowledge-based services, advisory assistance services, medical information technology, electronics, transportation. The idea behind the portfolios is we build knowledge in the portfolios of commercial business practices, good acquisition strategies so we get smarter within those portfolios across the department and across the federal government, on how we can buy services and construct business strategies to maximize the dollar return on our taxpayer money. So that is the purpose behind portfolios, so that we develop knowledge and understanding of common business practices, etc., within each of those portfolios because they are not all the same. That is the important thing to understand. The way you buy medical support services is not the same as the way you buy facility management services. How do we gain the knowledge and learn in this process?
What are some of the typical problems we have seen in DoD? Some of these will probably occur in your agency activities as well. The department is infected with a bad case of “cut-paste-itis”, where you take the last PWS available and say, “this is my current requirement”. Contracting comes back and says, “it looks like the one I bought 10 years ago!” So, analyze what a performance requirement and outcome is. Why is market research important? So that you're not buying the old technology, so you know what is going on in the marketplace in terms of technology, business practices, types of incentives. Believe me, you contract folks out there, being an old contracting officer, an RFI is not significant for market research. You have to understand both the requirement and the marketplace. Tasks with no performance standards. A task is merely a task to do for a contractor. If you do not have a performance standard that says how well the task has to be done, what is the contractor to do? In building a proposal to address what you're trying to achieve. Standards -- we will talk a lot about standards in just a bit. Too many personnel qualifications, or “I need 10 bodies”. Why do you need 10 bodies? “Well, I have always had 10 bodies.” I call it butts in seats. Your only deliverable is a weekly time card, I guess. And finally, business strategy decided before the requirement is really defined. The requirement defines the business strategy, not the other way around. A couple other things you may want available to you to download is the DoD guidebook for acquisition of services. It parallels the process I will describe in a minute. Also a new guidebook on market research. All of these are available to anybody to download.
Here is the acquisition process. Too many times we found in the department that people view this process from their perspective, like if I am a contacting officer, my only goal is to write a contract. Or if I am a requirements guy or COR, you look at it from your perspective. We need to step back and look at it as a work flow, as a process for acquiring services. It all starts with a mission requirement. The mission requirement takes a team of people, not just the COR, or the contract officer, or the program or contract manager, it takes everybody involved to build that team, understanding what you're trying to do, understanding the problems you have had, and do the market research to the first three steps are the planning phase. Get your team together, understand the requirement, come up with a common vision statement. We will talk about a vision in a minute. What are the problems, what changes do you anticipate during the life of this new requirement? Many of our service contracts run three to five years. Is your agency, your activity going to be stationary over that period of time, or will it be changing? What changes do you need to plan for in the way you write the new requirement? Those three steps are your planning phase. That forms the foundation for developing the requirement in step four. In step 5, you will build a business strategy to support the requirement. All of these build, one step upon the other. Finally, you execute the strategy with a new contract or order against the GSA schedule, or however you decide is best to execute. You cannot really do it without getting your team together, but defining that requirement is certainly within DoD, where we have had a significant amount of problems defining requirements. Finally, where we do another poor job is managing the results, step seven. So step 7 is also key. Your COR's also have to be on board so that you're delivering those mission results at the end of this process. It is a holistic process, from mission requirements to mission results, and everybody plays throughout this process, not just individual elements of it. It has to work together. It works together best as a team effort through this process.
Now, some of the things you do in step 1, form a team charter. Start with a vision statement. What are you trying to accomplish with this acquisition? We have done a number of “service acquisition workshops”, where we get a team together, contracting officers, COR's , and we ask “what are you trying to do?”. It's surprising how often they cannot agree what they are trying to do. If they cannot come up with a common vision statement, that is not a good harbinger for success. So, what we have done is you can download from the Service Acquisition Mall and download any of these forms. For example, this team charter. You can download the team charter, download a project plan, which is in either Microsoft excel or in Microsoft project, that has each one of these seven steps broken down in great detail. You can use whichever ones are applicable to your organization, and they are not so DoD-centric that you cannot use them. I have tried to build our tools and training as non-DoD as I can make it because it is generic enough that it applies to everybody.
This is just an example of a breakdown in step two in some of the steps that are involved. You can download the whole project plan from the mall. The Service Acquisition Mall is our attempt to try to provide that one-stop shop for knowledge and training on services acquisition. It is broken down by portfolios. It is http, so anybody can access it. It has been accessed from literally just about every country in the world. You access the mall to download our tool. I’ll show you how in a little bit.
Why vision? Here is the structure for how we define requirements. We break it down by your vision statement, supporting the vision statement are your high-level objectives. Under high level objectives our task statements or even sub-task statements supporting the task. It is like an organizational chart you see in an office. We did a workshop for the Arlington National Cemetery, and the director of the cemetery came in and clearly laid out her vision, the particular requirement for the grounds contracts at Arlington. Her vision was, "I want Arlington to look like Augusta in April." that sent a new -- that set a lofty goal for everyone working on the grounds maintenance. She said, “Here are my three high level objectives”. She said “I want every acre to look the same”, and she came armed with pictures about how every acre did not look the same. Second goal, “I want no damage to my monuments, because some of the contractors were knocking over the headstones and not taking proper care of the facility”. Number three, “I want respect for my services, because some of the contractors were trying to mow grass during a funeral, not really the image you want to leave with a family”. As we built the PWS, we could align all the tasks within the PWS with those objectives. That is the importance of going in with a structure in mind how to lay out your requirement. There is some talk about a Statement of Objectives or an SOO. A SOO is a good way to go. If you can lay out the high level objectives -- this chart implies the contractor in their proposal builds the task statements to support your high-level objectives. That is one way of doing a Statement of Objectives. The more common way most of us use are the Performance Work Statement, or PWS. This graph illustrates how a PWS is organized. Your vision statement, high level objectives, and each of the tasks broken down under high level objectives. This one, under ARRT tool, you can weight the high level objectives. Whatever weighting rubric you want to use, you can apply it in the ARRT tool. You can also weight the tasks in terms of the HLO. Clearly, you'd probably want to evaluate how they are going to accomplish those tasks or those high-level objectives you put the most weight on. That is one of the ways to use the weighting scheme in the tool.
What is this road map we're talking about? This is an example of what the road map is. Before we built the ARRT tool, we did workshops. A task basically consists of a result, a context, and an action. A standard for how well that action has to be accomplished. Standards basically come in three flavors -- cost, quality, or timeliness, and variations thereof. An AQL is Acceptable Quality Level. In other words, can you stand a variation to the performance of that standard, or does the performance of that standard have to be at 100%? Clearly there are things like environmental compliance, safety and health issues- I want 100% compliance with those. As for taxi pick-up, I could maybe tolerate some five or 10-minute variation in what standard I put in my contract. It all depends on the nature of the requirement as to do you or don't you have an AQL. Then inspection. What am I going to inspect? How am I going to inspect it and who is going to inspect it? Is there an incentive or remedy involved with this particular task? As you walk across this road map, you can see what you are starting to build. The task standards in AQL go into your PWE. You are developing your PWE, and with the weighting, you can decide what to evaluate in a contractor's proposal. You can also build your seed rolls or delivery items, reports as an example. That is the purpose of the road map, to provide and have an outline form that controls for the requirement and your Quality Assurance Surveillance Plan. We've seen Quality Assurance Surveillance Plans, and a lot of things in the PWS that are not in the QASP. One of the features of the tool, walking across the road map from left to right. Inevitably the Quality Assurance Surveillance Plan is explicitly linked to what you put in the requirement.
What is the road map? We call it the get‘er done tool, in terms of the structure for your requirement. What is a task? Tell me what the task is. Task is a result you want. It is a combination of nouns and verbs, not adjectives and adverbs. Nouns and verbs. Be as clear as you can, and this is the hard part. You see down there at the bottom, tasks have three components -- a result, a context -- in other words, what does that result apply to, and then what action do you want a contractor to take to achieve the result? Now, you might think that it's pretty easy. Well, I wish it was. You would be surprised how hard it is to focus on and define what a result is. We have worked with different teams. One team was working on an IT-type acquisition, and they were all IT people around the table. We said, ok, what is the result of this task? It took them 45 minutes of discussion to define what the result was. The good news is, it only took them 20 minutes on the second task, five minutes on the third task, and then they reprogrammed their brain on how to think about it. That's what we're asking people to do, to reprogram your thought process to focus on the results you are looking for, not telling the contractor how to do something. So that is the hardest part of the tool, getting people to think about a result.
What are standards? Standards convey the level of performance that I need to satisfactorily meet that task objective, that task outcome, that task result. Standards are going to be your cost drivers in terms of how well it has to be done. So be careful what you say -- what you set as a standard, and make sure it is mission critical and you really need it as a mission outcome in terms of the level of performance. Make sure it as clear and measurable as you can make it. Because if it is not measurable and not clear, you are going to run into trouble in terms of expectation management with your customers. Been there, done that. Standards described as adverbs and adjectives.
Ok, here is a typical performance work statement example. “Contractor shall perform and document initial inspection for newly received vehicles, equipment, actions shall be completed within 72 hours of equipment arrival at the station. Documentation shall be in accordance with coming in this case, Army regulation 73-5.” Now, what is the result we are looking for? I use as an example in the classroom, and since you are all out there and cannot give much feedback other than what this camera is looking at me What would the result be in this statement? Ok, and what is the context of that result? In other words, what does the result apply to. And then finally, what actions are we asking the contractor to take in terms of achieving the results? Look at this one for a minute. See if you can figure out what is the result. The result is not “vehicles and equipment”. I will give you that much. Let's look at what it is.
Let's break down the pieces. The result I am looking for is the initial inspection. That is the result. What does that inspection apply to? Newly arrived vehicles and equipment. What actions do I ask the contractor to take? To perform and document those inspections. My standards are to complete it within 72 hours and do the documents in accordance with regulations. Those are my standards. As I look at a statement like this, this is what would appear as the task statement, this is what would be in the PWS as the complete statement. AQL is a variation to the standard. Is it something that you can allow that variation? As I said earlier, there are some things you cannot allow. The requirements development team, along with contracting officers, are the ones who need to think through that and determine whether or not an AQL is applicable or not.
So what is the PWS structure? This is another thing we are striving for in the department. The FAR does not have a structure for what a PWS should look like. Consequently, the Department of Defense is all over the map. I suspect your agency or activities may not have a common structure for a PWS. What we're trying to do with the ARRT tool is to develop that common structure across the federal government. From a contractor's perspective, they do not like having to look all over a document to find what is applicable because we tend to hide it in different places. If we can be consistent with industry and how we describe our requirements, we are consistent in how we organize a solicitation because the FAR requires that. But the important part of the solicitation is a requirement. If we can standardize how we write our requirements and structure them, industry will love it and not waste as much time searching for information within a PWS. So the way we have organized it with the ARRT tool is your introduction covers your mission, your background, general performance requirements in part 2, specific performance requirements in part 3, which is what the ARRT tool is really designed to help you with. Part four is the special requirements, in terms of government furnished properties, special securities requirements, the deliverables in section 5. Related documents in section 6, and the performance requirements summary, which is a table outline of all the requirements in the PWS along with remedies and incentives. So that’s the structure of the PWS in the ARRT tool.
I talked about the PWS. What about the Quality Assurance Surveillance Plan? How does that work? If we use the ARRT tool to build the performance task standards and AQL-- that is my green box, blue box, and purple box in terms of the actions, results, and context and the standards. They drive what you're going to inspect. The “what” should derive from the results you're asking for in your tax statement. The “how” is going to be dependent upon the standards that you have set and the outcomes that you're looking for, as well as what you have asked the contractor to do as part of the requirement. Finally, the “who” falls back to that good old COR that will inspect all this. If we used our example from earlier, equipment inspection, what am I going to inspect? The results I wanted were initial inspection, so I had better look at the receiving reports. The standard that I used -- I am looking for the inspection reports, and the submittals from the contractor. The “how” is do a random sample of those documents and see if they were submitted within 72 hours, and were they in compliance with that regulation as the “how”? And then the “who” is going to be the COR or the technical representative that will get this equipment. All of that goes into the Quality Assurance Surveillance Plan. Again, within a QASP, there are no standards across DoD, so we built a standard format for the Quality Assurance Surveillance Plan.
So the PWS I've got here is the key communication device with industry. It is truly the most important document, the most important when you will build through this service acquisition process. So be as clear as you can. List the requirements in descending order of importance. Use drafts if you have time so you can get comments back from industry. Again, focus on results, not body count. It is the results you are buying. Let the contractor figure out how many people it is going to take. Our challenge is to accurately and artfully define the results we are looking for. Because if you do not, one of the pitfalls we have learned certainly through my experience in the government is this little legal principle called ambiguity will be construed against the drafter. Which really means if you are not clear -- and it can be subject to interpretation -- and somebody interprets that differently than you do but you are not clear, that is an ambiguity and that will be construed against you, and we will usually find it in favor of the person who made the interpretation different than yours because you were not clear. It is a longstanding legal principle. It again underscores the importance of defining what it is we want, what the results are and not the body count. If you can do that, you can avoid the protests that would come. And that will delay your acquisition. Avoid claims and misunderstandings later on. The last comment in here- I used to work for a general when he was the head of Air Force contracting and his contract was that the best contract in the world cannot fix a poorly defined requirement. That is the importance of what we're trying to achieve with the ARRT tool, to help you and give you a tool you can use to help build a better requirement.
What the ARRT tool is is basically a Microsoft Access file. You have to have Microsoft Access. One of the key requirements I laid out early on is I did not want the government to have to buy any new software. Most of us have Microsoft Office, so if you have the Office suite, you have Access. If not, you can get it. The ARRT tool itself is just a file. It works off of your computer. So you have to have Microsoft Access to launch and operate the tool. We will talk about that. When you download the tool, it will come with an 80-page instruction manual to provide instructions on how to use the tool. There are training videos within the Service Acquisition Mall to help you with it as well, plus we are available at DAU if you have questions. It builds the standard template, build your PWS, your PRS, and Quality Assurance Surveillance Plan. We are using that process as a work flow. If any of you are familiar with lean, or lean six sigma, one of the principles is “define a common process” or work with standard work. We define the standard process, steps 1 through 7. The mall is intended to foster and facilitate step one in terms of providing training, access to the ARRT tool, a skill development center on the mall. There is even a game for defining results, context, and action. I will show you what that is. So we have built the ARRT tool as far as defining the requirements. That is one of the biggest problems with DoD. We are also going to start ARRT class. I have a question mark for December 5 because I have been trying to start the class for a month or so now. It will be much like we're doing here, taking a particular aspect of using ARRT and explaining it in a lot more detail. We will respond to questions from the user community in terms of what should the next ARRT class focus on.
Why should you use ARRT? Why is it important? This is a survey we did back in March. It asks several questions, but these are the two more important questions to me. “Did it help you build a better requirement?”, and ”Did it help you do it faster?” As you can see from the response we got back in March, yes. You can see the question there. “The ARRT forced me to think carefully about the wording to produce specific, focused tasks.” I am sharing this chart to show you where we are going that DAU in terms of developing other tools that will leverage off the other information in ARRT. Step 7 is a challenge for DoD, so I want to create a performance assessment tool that will help the COR's do their job, create their inspection checklist, their timing of inspections, based on the use a random sample, monthly inspections? It will capture their data, due random tracking, and then build their monthly tracking charts. At the end of a 12-month period, create a submission, create their CPARS report for the end of the month -- the end of the year, rather. The other tool we’re building is a Plan Assistant, and an IGCE tool based on what is in the PWS. This is a description of the tools. I'm going to skip that. Why don't we take a break now. You have been very patient to sit through 40 minutes. Let's take a short break, and we will come back and I will show you the ARRT tool and how to use it.
LE: Welcome back to talk about the ARRT tool, the Acquisition Requirements Roadmap Tool. There is a box down there at the bottom of your screen, if you have any questions that you would like me to address, I would be happy to. Just type them in there. We have some diligent fingers back there capturing all those questions. They already have a long list for me to answer.
Ok, how do you get to the ARRT tool? First off, go to the Service Acquisition Mall. Let's go shopping at the mall for a minute and see if we can find any bargains. Our mall was created -- you can kind of make fun of it, but it is that central point for information on service contracts, services acquisition. As you mouse over, it tells you what types of information are available in different wings of the mall. There is a skill development center. I will show you more about that. All the videos are on the video kiosk. Each of the steps has a narrated video that is basically narrated PowerPoint that talks about what each of those steps is, talks about the generic work products from a team charter to a communication plan to project plan to stakeholder analysis, sample forms you can download. That is information. Here are some of the ARRT videos that are available. All those are available in the video kiosk. Skill development center is basically a list of courses. This is a new thing, the ARRT game. It tries to work you through how to define outcomes, results, context, and actions. For example, analyze, define, architectural baselines for the program office. What is the result I'm looking for there? Probably an architectural baseline, right? I got that one right. You get the idea. You can go through and improve your skills at defining -- because as I mentioned earlier, this is the hardest part of the process, getting people to define those results.
How do you get to the ARRT tool? There is a little button on the side of the mall map that says ARRT tool. Click on that, read the information in here, and the button in here says “click to download”. We have tried to make this very hard, failed miserably. What we need is your first name, last name, phone number, e- mail, and organization. The reason we're asking you to do that is so as we update the tool, as we provide new versions out there, we send out a blast e-mail to all registered users saying that a new version is available and you can download that. Also when we start the ARRT class, we will send a blast e-mail out to registered users. Other than that, there is no cost for using the ARRT tool. There is not any cost. It is developed by the government -- not by the government, but by a contractor under government direction and process, so we own the data rights. Anybody can use it free of charge. Never mind, I will not register that online. It will send you an e-mail. Then it tells you how to download the tool. Click on it and download the tool. It will come with the 80-page instruction book, and I think it is, with 2.0, we have also included a PowerPoint chart on what’s changed if you're using one of the earlier versions. That is how you get to ARRT.
Now let's go to ARRT. If I can get the technology to work. Here we go. This is the main page of ARRT. When you download it, you will need to save it to your hard drive because it will not work in an e-mail. It will not work from an e-mail. Otherwise you will ask lots of questions, why didn't it work? There are several things on the front page. When you open it the first time, it will ask you to enable the content of the database because it is basically a database program. That is all Microsoft Access is. You can create and manage a project. You can configure the ARRT settings. We have created that common template I mentioned about a PWS. Having grown up in DoD, I'm tired of having tools forced down my throat that I cannot change anything in, so we have made this a fairly flexible document. What we're trying to do is work toward that common interface with industry so industry knows where to find stuff. You can identify who the current user is; in fact, with this version, we are on version 2.0. We launched 1.0 in August of last year, followed by 1.5, 1.6. We launched this version in early October. We are continuing to develop and refine the tool. As I click on “create and manage a project”, you will come with, with 2.0 has more in it than I have in this version here. But “maintain the Clark river” is a dummy example of a dredging requirement that we included just so that you can see how to navigate around the tool. At the top there is a project list, which is what I’m on now. Project information. If I click on that, it says select a project. Go ahead, and maintain the Clark river. The name, when it was created, and a progress chart in terms of where am I at in completing this document? My task statements, my deliverables, my PWS and my QASP. If I go back to the top again, here is my project -- maintain the Clark river. My high level objectives -- I have two: Make the river navigable, identify the navigation channel. I have three tasks under “make the river navigable”. Establish the depths, the width, and dispose of the waste. You can see the yes’s and the no's over here. That “N” means I am not done with-- I have to go back, think about it, get other opinions. That is something we have built into the tool to help you keep track of where you are at. The arrow buttons are ways of moving information around in the tool. For example, if I wanted to take the “collect and manage waste”, rather than make that a task under “make the river navigable”, I want to make that a high-level objective by itself. I can click on the arrow button over here, and that moves it over to become a high level objective. Or maybe I want to make a sub task under the width. I can click the arrow in the other way, and it becomes a sub-task. The database takes care of organizing and numbering all the tasks, unlike Microsoft Word. All of this exports to Word, but we recommend that you work in the tool until everything is set the way you want it because it is a database. You can move stuff around easily and it renumbers everything. Then once you are done, export to Word, and send it to contracting for folks to review as a Word document. We are also working on XML standards for PWS and QASP. A contract writing system, for example-- you can import it as an XML file.
I clicked on “establish and maintain depth”. What came up were the statements that were already created in ARRT with respect to maintaining the depth of the channel. Some additional information, as Paul Harvey says, the rest of the story goes in here. Under standards -- what are my standards? 12 feet deep below water? My AQL says I can stand some variation to 12 feet. Sounds like it was written by an engineer. That is the purpose of an AQL. If I could not stand any variation, it would be blank. That implies that conformance to standard is 100%. How am I going to inspect it? What am I going to inspect? The depth of the channel submitted by monthly reports -- how? I will let the COR take the boat out with the 360-degree sonar and GPS and do a random sample along the river every once in a while to verify the depth. You could be as specific here as you want in terms of this is your dialogue with the COR and how you want this inspected. What are the deliverables? In this case it is a monthly status report.
So how do I go back and create a task statement? Well, let's look at “identify the navigation channel”. I have asked him to mark the channel, maintain the channel markers. Do you know what I forgot? I forgot to ask him to modify or update the navigation charts. Let's add with the “add” button here, a new task. This is probably the most probably the most important screen. My high level objective up here – “identify the navigation channel”. My task statements -- I have identified what this statement is. What am I looking for? I have gone to the contractor. Now I need him to update the navigation chart. What is the result I am looking for? What I am looking for is navigation charts, right? If I could type and talk at the same time, I would be dangerous. Now, if I can deviate for a minute, we built into the tool, under sample results. Remember I said knowledge-based services, or ANAS was the biggest consumer of services? If you need admin support, what are some of the common high-level results under admin support services? You can use this as a shortcut or a thought stimulator for defining those results. Engineering and technical support. Acquisition management. Facility-related services. Financial management. The tool already contains a fair amount of information. But remember, it is a database. As you are entering information in your copy on your machine, it retains that information. You put it in, it will still be there. Let's close this.
So, navigation charts, what do I need? What do these navigation charts applied to? What is the context of those charts? In this case, it would be the Clark river, right? Now, what actions do I want to take as pertains to those navigation charts? Again, under simple actions, we have a whole list of verbs here, and these are not mean to be an exclusive list. These are thought stimulators. Think about it. Analyze what is the requirement is you are looking for? The results that you need? In this case, what do I need? Is update in here? Ok. So, let's close that. I want to update. Maybe I want him to “maintain”. Is maintained in here? Yes. So, I can hold the shift key down, the control key down, and -- not the shift key, the control key. Maybe I want him to “produce” them as well. What else? “Track”? So, I can continue to add those verbs in here, maintain, update, produce, track. I can move those up and down. I can move this all the way to the top, all the way to the bottom, or delete it. The thing is to put them in the right order. What we have tried to get the tool to do is to build the PWS statement for you. If you’ll notice up here in the blue section, it says, “the contractor shall maintain navigation charts for the Clark river”. It does not always get the grammar right. I have to admit that. If you are happy with this, or you can go back and change that. For example, if I can go back to navigation and charts, add an ‘s’ on that. It has an ‘s’ on it. This is typable text. I can adjust the grammar. Focus on the questions, A, B, and C, and let the tool do the work for you in terms of building the statement. There is a box. It does not look like a very big box, but there is no limit to the text information you can put in there. Use that to define if you need to expand on what you really mean by that. Use that box for the rest of the story, as Paul Harvey says.
If I am done with my task statement, the right hand corner with the arrows is to get you to the next step. It says, “you changed what me, the computer did”. I say, “Darn right. I wanted to get the grammar right.” I mentioned sample standard. They come in three flavors. Cost, quality, and timeliness. For the navigation charts, maybe there is a compliance aspect where I need to be compliant with federal laws. I can edit this. Let's add that. We also have a button in here, if you used a previous standard, you can use it again. Now, I want to edit that. And say state and US Coast Guard. So, again, we have tried to make it as flexible in this case of the compliance standard. I want 100% compliance, so I want nothing in the AQL. I have one standard now for compliance. There may be others for complete, accurate. Certainly for navigation charts I would like to have some degree of accuracy to them. When do I want them delivered? A timeliness standard. What would be my time, my schedule for delivery? Again, these are not going to be 100%. Maybe not 10% to what you need -- but they are there for thought stimulators to say, ok, what would be a good schedule for those navigation charts? Is 10 days after dredging sufficient? If it is, that becomes your standard. Use the tools and samples as thought stimulators and discussion stimulators for your acquisition team. I will use that one statement as my only standard so far and go on to the next step.
You see this chart, the thing between the blue arrows that looks like a scroll? What this is is your work flow. What you are at in the process. It will take you anywhere to where you are at the process for that particular step. So, if I go through now and say, ok, what am I going to expect? All the standards -- if I had multiple standards in here, are they all tied to this one inspection methodology? I may have one inspection methodology for one standard and another for other standards. So, this gives me the flexibility of defining which inspection methodology applies to which standard. In this case, I only have one, so I am going to assign that to that inspection methodology. There we go. What will we inspect? What is our result? Navigation charts. Well, I better inspect the navigation charts. And we have a convenient button here that moves the navigation charts from your results to what you inspect. Here are all my current deliverable out here, one through four. Do not see navigation charts. Better create a new deliverable. It numbers them automatically for me. So, navigation charts. It is now a deliverable. And down here in the description, back in DoD, we use seed rules. You can expand on what the deliverable is. In the interest of time, I am not going to do that. I have created a fifth deliverable and it is tied to this task. Right now if I was to look at the PWS, it would be in the PWS associated with this task and also under section 5 under deliverable.
How will I inspect it? My standard is compliance. How- I am going to ask the COR to verify the charts. Regulations. Now, could I ask the COR to inspect the accuracy of the charts? Well, if I want to, I have to go back in and put that as a standard, because I do not have accuracy as a standard yet. This step is somewhat of a cross check between what you want to inspect and say “Did I really want the contractor to do that, or put a standard in for that?” Make sure what you are inspecting aligns with the standards of performance you have asked the contractor to do. Who is going to do it? Well, my friendly COR. Here we go. Government representative. I am going to add that. And is there an incentive or remedy? This is one of those blocks where if the contractor fails to update or comply with regulations, what is my remedy? If it is a fixed price contract, I can take a deduction off the price of the contract, or I can put in here, the contractor will re-perform the navigation charts at no additional cost to the government. If you have a cost contract, you need to be careful because the contractor may bill you for that again. Use your best judgment as regards to remedy, but if the contractor fails to meet the best performance standard, the government is entitled to a corrective action or remedy for that nonperformance. Because we are paying him to perform against the standard set in the contract. This tool goes into the PRS which goes into the PWS. When the contractor makes a bid against the PWS, he knows what the penalties are if he does not perform. That is why it is contractually binding. So, as I complete this task, it gives me one more view of what am I going to inspect, how am I going to inspected, who is going to inspect it, and what my standard is. The only standard I have written so far – “comply with regulations”. So, I close that.
You see the boxes up on top? These little boxes here, standards are complete? These are the boxes that take these from no’s to yes’s. Have I completed that task? I click the box and it goes from no to ‘y’ for yes. That is a way to control or account for where you are at. Now I can weight those requirements. In this case, this is what my weighting looks like. I want to look at the whole project. I have “make the river navigable”, and “identify the channel”. I can weight the tasks within the high-level objective. I may want to take the importance of that one down just a little bit. Again, what you would use that for is looking at what you want to evaluate from a technical evaluation standpoint. When you're done with the weightings, here are the work products that the tool will generate for you: your PWS, the PRS, the summary report, your statements, past statements, standards, inspection methodology, and deliverables. Close that. As I create a PWS for this project, this is what we talked about earlier. The vision statement, the introduction, a background if I had background to put in here, what the scope is. Notice when I click on "scope," these are all typable test fields. Yes you can cut and paste. We have created areas for standard information. These are things that we have pulled out of common PWS's. You have default text in here so you cannot just use it without thinking about it. You can also go to the top. If you do not subcontract management, there is no subcontract. Unclick the box. Take the weighting out so 2.3 is still contract management administration. If you do not use it, there will not be text there. We are trying to get back to standard formats for all government agencies. Now we get down to section 3. “Make the river navigable”. What my task statement is, what my standards are, what my deliverables are. You can see whether it is complete or not. Again, these are fairly short, cryptic statements. That is why he may need Paul Harvey's “rest of the story” to describe what this task involves. In that description, don't compromise the clarity of what you're asking the contractor to do, the results you are looking for. I am going to go back to the top here. If I want to export this whole PWS to word, if I have done a little cutting and pasting, I can clear all the formatting so that all the fonts will appear the same when it gets to Word. Notice it has a table of contents. You can unclick that if you do not want them.
And here is my PWS. It comes with an index. And basically what you saw on the screen earlier in terms of the section 1, my vision statement, introduction, background, section two, and section 3. We had a large air force activity use this tool to develop, I think it was a $600 million satellite tracking station. The comment we got back after they issued the RFP was how few comments they got. Because it was clear. Because they used the tool to force the discipline and the thought process to be clear in the results you are looking for. Leave the “how” up to industry. It does make a difference; we’ve got Army sustainment command out of rock island has been developing templates for, and logistics requirements at Army installations, and so they have basically three: Transportation, base maintenance, and supply. All their 70% of their PWS's have the same standards. They are adjustable and have the same features for the PWS. That is one of the benefits of using something like this. These buttons over here, if you are the originator of the tool, you can lock it so nobody else can make changes to it. So that if you wanted to create it and send it to a colleague to review, he could review it, but he could not make many changes to it. How is he going to give feedback to it? You have to click on the yellow stickies. If I send it to my colleague to review, he can comment on this by creating a new comment. He can make comments at the project level, a task, standard inspection, PWS. He would write his comments in here and submit. So when he sends that document back to you -- ok, thank you. We have spell check in here. I am not a good typer. It catches me all the time. So, anyway, you send it out for review and comments, get those things back, and know exactly where that comment is tagged to with the yellow stickies. And let's see. Lockable. You can go down -- you can move to the task level and the sub-task level and even sub-sub- task level if you need to. “What” starts to become “how”. Try to guard against writing requirements that define the “how”. Leave that up to the contractor. We're trying to define the result. Let's take a short break and we'll come back for Q&A.
LE: Welcome back. You can probably download these slides within a day or so after this podcast. There was one question about future upcoming events. Upcoming events might be premature on my end, but this is the chart that outlines the tools that we're in process of developing a requirement we can complete for. Basically this section. The section development, the performance assessment tool, a cost estimate tool which takes what you put in your PWS to come up with a good cost estimate for this requirement. All of these will be putting information into the database for the requirement. So, we honestly do not anticipate even starting to test- the first one will be the performance assessment tool and we probably will start on that next summer and be in a position to do beta testing in the fall of next year. These are still a ways out yet.
I have several questions, I have some good questions. Just a rehash on what an Acceptable Quality Level is- this is a variation to the standard that you put in your PWS. If you leave the AQL blank, that means that the contractor has to perform 100% to the standard in the contracts. If it is a health and safety issue, a security issue, you are darn right you want 100% compliance. If there are areas where you can give the contractor latitude, where 100% is not always required, that is where you would specify the AQL; or, another way is to write the standard in such a way…one of the other questions was what is the difference between a AQL and max error rate? An error rate is something that you can write the standard in such a way that you do not need the AQL. The AQL is covered in the way you wrote the standard. Ok.
Are there plans to include a way for the system to develop evaluation criteria? That is really what the Technical Evaluation Plan (TEP) assistant is for. It is based on those weightings you inputted, as well as those evaluation factors to help the technical team decide what evaluation factors they want to use in section M of the RFP. And that will help the contractors develop proposals. So, it is all related. Basically, ARRT -- it is built into ARRT.
Let's go back and look at ARRT for a second. If I want to create a new project -- let's start with a new one. Project name? Let’s put “Test1”. I have to come up with a vision to start a project. So, -- “enable the workforce to develop better requirements”. Ok? That is my vision. So, it is new. It was created on this date, version 1.0.. I will use the default PWS template and I will use the default QASP template. Notice, that is what is in here. I could create my own PWS template. Notwithstanding what I said earlier about standardization. We are trying to standardize. But again, if you must, you can create your own template. Hit the save button, and then I would go on to define my high level objectives, my tasks. I mentioned how to configure the ARRT settings, which is the second button in here. I can manage the WBS nomenclature in here. If you have a specific way you number your deliverables, you can set that so the tool will use that. If I want to start with how many digits in the numbering system, again, we built flexibility into the tools so you can do that. The same thing with PWS sections. You can configure the tool in a variety of ways. We worked with DISA, they changed the standard to work with the tool by making minor changes in the PWS. Now their standard is acceptable to the contracting folks, everybody wants to see the PWS. So, again, building of flexibility into ARRT. The upcoming attractions are the required standards.
"What do you mean when information will be ‘reserved’?" For example. If I go back to “maintain the Clark river”, if I want to say, business relations, I do not need that. I am going to take it out. So when you get back to business relations -- where did it go? It should just say reserve. It disappeared on me. It disappeared on me. It should say “business relations” and a ‘reserve’ in the text. If I click on this button -- there is a button up here if you need to add sections. Put the section name, the section title, the description of what is in that section. Again, you can paste it in little blocks like this. The information shown when you need to review it -- if you put in the reserve catalog -- category or deselect it, it will not be in the PWS when you export it to Word or anything else.
However, this agency prohibits us from downloading anything to our government computers. Do you have any suggestions for how we can download this tool? We have run into this with other agencies as well. You can -- if you can find someone who can download it to a central server and give people access to it, you can do it that way. The only difficulty then would be you do not know who has copies of the tool, so when e-mail comes out, you may not be aware there is a version 2.5. I looked at version 2.5 with a developer down in Fredericksburg. 2.5 will be a SQL server version or have the capability to put it in a server-based application so multiple people can have access to your work or you can restrict however you want to restrict it. That will be out probably in the second quarter of FY13.
What is the web address for the mall? http://sam.dau.mil. You should not have any difficulty accessing it. It does not require a password or anything like that. If you want to get access to the tool, click on the service acquisition mall and click on the prompts that will tell you how to get access to the tool.
"Where can you find the guidebooks for services and acquisitions?" This gentleman left his e-mail. I will certainly send him up links to it. If you download the slides, it has the e-mail address on the slide.
"Will projects be upgradeable?" You bet. Notice there is an import button here. If I hit the import button, I can do that. If I hit import, I browse. In this case there's probably not another picture on this computer. From our experience, we try to make this as flexible as possible so as we update versions, all you have to do is import your old version, let's say from 2.0 to 2.5, it will come straight across and be updated in the new 2.5 version. We have made the change many times. Ok. Here is a good question.
"The FAR basically provides two-ways to perform service acquisition; one using a PWS and one using a SOO." Absolutely correct. The government-provided PWS appears to essentially be the process of government defining the solution to which all vendors must propose. What we're trying to use the ARRT tool for is to define the results that we need. And perhaps that structure, for example, under high level objectives, if I wanted to use a SOO- Arlington National Cemetery for example. You can let the contractors come back and say, how are they going to not damage the monuments? How are they going to respect these services for the employees that are out doing the work in the cemetery? A SOO is a good way, a perfectly acceptable way of letting the contractor have more freedom, if you will, in defining how they will match your objectives. PWS gets into, yes, a little bit more specificity in terms of key results under an objective. Again, it depends on the nature of the requirement, and how specific you're going to be. For example, grounds maintenance, I could probably give the contractor a pretty wide amount of latitude as to how they are accomplishing grounds maintenance. If it comes to aircraft maintenance, I will be very specific with the results and what compliance I want with respect to maintaining these aircraft for safety of flights, compliance with manufacturers' technical orders and what kind of periodic maintenance programs are required. It all depends on the requirement, whether it is best suited for a SOO or a PWS. That is up to the acquisition team, the contracting folks, all working together to come up with the best solution. Both are perfectly acceptable, but PWS should not define the “how”. It should define the results. That is why we have built the tool the way we have to get those A, B, C questions. It still gives the contractor the latitude to come up with the “how”, so you are not defining a common process that everybody has to bid to. You do want contractors to innovate as to how they meet your performance results. So, good question. All I can say is stay tuned. If for example, your agency might like to be a beta site or a test site as we work on these additional tools, I would recommend you send me an e-mail or send Jeff over at FAI an e-mail and say, “hey, we would like to be a beta site”. These tools are available to all agencies. In fact, I know some consultants are using it.
Somebody asked about can ARRT be used on a mobile applications such as and iPad. I want to say, man, I wish so. Right now ARRT is written in Microsoft Access. Part of why we did that is a lot of people have Access and we don’t have to buy new software. We want to make that tool something that can be used on a tablet device. Whether it is an iPad or somebody else's tablet. We want to make it truly mobile so the COR can have a mobile device, go around, but the tool to the organizing and the charting of performance over time in the tool. The right now, no there is not an app. Unless you want to run parallels or some other device that will let you run Windows on an Apple, and I am an old Apple guy, there are no Apple applications for this simply because it is Microsoft Access and it is not on that platform no. But stand by. We will not be deterred yet. And again, those in the agency who said, hey, a lot of organizations prohibit people from downloading, there are ways. We have used the ARRT tool behind classified firewalls. It goes through the Navy system. Again, it is just a file. There are no executables. That seems to be the big bugaboo with the IT folks. There are no executables. You already have the program. It is just a file. It is like if you were to attach a very large excel spreadsheet to an e- mail and e-mail it to a colleague. As long as we keep it that way, it usually goes through networks very well. Ok, any other questions back there? Uh-oh. More questions. Ok.
What about the schedule? Can you create a schedule from this? Not yet. If I was to take you to the Service Acquisition Mall -- let's see. By schedule, I assume you mean an acquisition schedule. I'm not quite sure. But if you go to the mall and look at form the team, one of these steps in here is the project plan. If I go to project plan, here's the video that talks about the project plan, and here is my downloadable Microsoft project or excel project plan. I do not know if I will blow this thing up if I try to download. I will not download it. It will create your charts, as opposed to excel. If you want to create a schedule for your entire acquisition, this is where you would go. Again, what I would remind you is where we're looking at in terms of drafting that schedule is this process flow. It will start from getting the team together to completing your acquisition and managing your results down here. Everything in here has a detailed breakout of each one of the steps as to who is responsible for the action. As you use this chart, you need to put names in here so everybody knows who is accountable. That is the purpose of establishing that team charter. Getting everybody to agree on what your objectives are. If you download the team charter here in the mall, this is actually a team charter used it at a team out of Arizona. They wanted to create a requirement to have a contractor come in and evaluate another contractor’s safety plans, because it wanted to use arranged to test weapons. So, the vision statement is just what it says on the slide. Create the safest working environment for testing the most dangerous systems in the world. This is not where the vision statement started. When we got started, the team was sitting around the table, just like this conference table right here, and we asked “what is your vision for this requirement?” They responded, “Environmental engineers, safety engineers”, and they were focused on the people. We said, no, that is not your vision. It took them an hour and a half. An hour and a half to come up with this vision statement. But once they got that vision statement, and everybody bought into it, whether you were engineers, safety people, a contract officer, COR's. They all marched to that vision. Then you see the mission statement down there. Believe me. Getting a team started is almost as hard as defining that requirement, because you have to get everyone on the same sheet of music, so everyone knows what their part is and they do not create a bunch of noise. So the project plan and the team charter are key documents to make sure that everybody is on the same page, and use a schedule to say this is the way we're going to move.
"Does the ARRT tool help capture the higher level mission or purpose?" Yes, it does. When I go back to use the tool, and I go back to close my import box. Here is your vision statement. It is a typeable text field. You have to fill this in yourself. The tool will not know. That is what is important. You capture that. Then it becomes part of the PWS and goes to the Quality Assurance Surveillance Plan. You capture once, use many times. That is what our objective is in part, capture once -- or a place where you can cut and paste, and, consistently-used information so you do not have to do it again and that will transfer through various documents. What I did not show you was the Quality Assurance Surveillance Plan. So, if I look at QASP. It brought over my mission statement and my vision statement. It talks about what the purpose of the QASP is, the authority of the QASP, roles and responsibilities. The contracting officer's representatives. Method of surveillance. Let's go back and look at the surveillance matrix. And here, in essence is the COR's inspection checklist, which is part of the QASP. There are lots of discussions in the DoD all the time.
Do you make the QASP part of the contract? Our answer is no, you do not. Why not? You already have this as part of the PWS. If you make the QASP part of the contract, then every time you changed the QASP, you have to make a new contract. You may have to get the contractors to agree to that change. There is no need for that. The QASP is an internal government document; it should not be part of the contract. Any performance requirements not in the QASP or that is in the QASP, but not the contract, the contract is what governs. That is why we created this tool that takes what you have written in the PWS, puts it in the QASP. If it is not in the PWS or the contract, you really cannot inspect it. You may try to inspect it, but you cannot hold the contractor accountable for it. Ok.
"When someone yellow sticky comments, do they put out a word tool or do they simply send back the ARRT tool?" They would add just that and it would say, I've looked at that and the comments are embedded in the tool is the way it works like now. Right now the ARRT tool does not allow you to merge your comments. You’d do that manually. You take your comments, acknowledge them, and adjust your section, or just whatever section of the PWS it would apply to. So, we are about to run out of time. If you have other questions, you can send us e-mail, if you have issues, or if you can download the tool. We can definitely work with you. Our objective is to help the government write better requirements to get better proposals from industries to get as much value as we possibly can from our service acquisition dollars in what is going to become a much more budget-constrained environment. I wish you all well and good luck and use the ARRT tool and let us know your results. Thank you.
