Press Briefing by Under Secretary of Defense Lord and Defense Innovation Board Chairman Schmidt on Software Acquisition and Development Practices

ACTING ASSISTANT TO THE SECRETARY OF DEFENSE FOR PUBLIC AFFAIRS CHARLES SUMMERS: All right.  Good afternoon.

It's good to see many of you back in the room, busy day today at the Pentagon.  And I want to thank you again for being here for the department's introduction to the final report of the Defense Innovation Board's Software Acquisition and Practices Study.

Under Secretary Lord and our distinguished guests from Defense Innovation Board are here to highlight the -- the importance of software and to detail what the department needs to maintain our technological advantage for decades to come.  Each will provide opening remarks and then Lieutenant Colonel Andrews will facilitate the questions.

And we have a hard stop at 1:05, 13:05.

And with that, ladies and gentleman?

UNDER SECRETARY OF DEFENSE ELLEN LORD: Thank you, Charlie.

Good afternoon.

This morning, my office formally delivered to Congress the final report of the Defense Innovation Board's Software Acquisition and Practices Study, or SWAP.  The report, I think, is an energizing product and coincides with renewed and palpable momentum to quickly evolve our approach to software acquisition and development.

Implementation of some of the study's recommendations, such as the creation of new acquisition pathways for software and a new mechanism for authorization to operate reciprocity, are already under way.

The department appreciates the efforts of the Defense Innovation Board, the internal department working group that supported the study and many others who contributed their time and expertise to the creation of the report.

Its format, fittingly, differs from past study reports the department and Congress have commissioned on software and technology acquisition challenges.  It is a remarkably accessible product.  Its recommendations are time-bound and, most importantly, it includes draft implementation plans, complete with proposed lead stakeholders, which my staff and I are already iterating on.

I plan to provide an initial implementation plan to the defense oversight committees within the next 60 days.  And then the OSD team will continually implement and iterate on that plan, providing full visibility to all stakeholders.

Defense technological advantage today is enabled by hardware, but its capability is defined by software.  There is an undeniable urgency to develop and deploy software faster, faster than our adversaries, in order to maintain strategic and tactical advantage.

As Dr. Schmidt aptly noted in his forward to the report, that it is, quote, "scarcely possible to imagine how the department can achieve the modernization objectives of the National Defense Strategy without overhauling its approach to software."

I am committed to sustaining the pace of change, beginning with A&S strategic priorities.  A&S is taking a comprehensive approach to software modernization and, in partnership with the services and other OSC elements, will action the SWAP recommendations in conjunction with the most relevant recommendations from previous reports, including the Defense Science Board's Design and Acquisition of Software for Defense Systems and the 809 Panel report.

DOD cannot achieve the full breadth of its software modernization priorities alone.  We must partner with the private sector and Congress to enable delivery of world-cost -- world-class software at the speed of relevance.

The department has the authorities it needs to modernize its approach to software development, but navigating the current system is too onerous.  The department calls upon Congress to provide explicit and specific authorities for software, such as a new appropriations category to fund software as a single budget item.

With that, I'd like to hand it over to Dr. Eric Schmidt.

DEFENSE INNOVATION BOARD CHAIRMAN ERIC SCHMIDT: Well, thank you, Secretary Lord.

When you started a little bit more than a year ago, this gap in software was one of the first things that you noticed.  And at about the same time, the Congress actually wrote into law that we had to do this report.

I have famously opposed doing any reports because I assumed nothing would happen.  So we've been doing, sort of, hand-to-hand combat and engagement internally to try to change, as best we could.

So we embarked on this report to essentially identify a huge shortcoming in the DOD, which is the modernization of its software.

And when I come here, I think of the software as being, like, 1990s software.  It -- it feels, sort of, of that generation.  And I've done this a long time.  And it worked pretty well back then, but it isn't appropriate for now.

And what I'm particularly proud about was that the study was done in the open.  And so you can see not only the pieces of the study as it was developed in the open and with lots of feedback, but you could even see the comments from many, many commenters, many critics and many stakeholders.

What's interesting is that, in software, you have a normal process of iteration.  You have a:  we try this, we try this, and so forth and so on.  It's as much art as science.  And that process is not available to the software people inside of DOD.  The historic acquisition process -- which is, sort of, requirements, procurement, wait, deliver and -- and turn on -- is not how software is built.

And so it became fairly clear to our team that it was necessary to have legislative changes as well as operational changes, which the Secretary Lord was already, sort of, thinking about.

And so the reason to have a report is that it takes two to tango.  It takes the leadership on the DOD side and on the Congress.  The legislation has to reflect modern software practices.

So it seems to me that these -- these recommendations are transformative in that the -- the groundwork that we're laying should allow everybody to, sort of, operate in a more modern way.  

And if you're confused by that, you say, "Oh, yes, it's just another report," remember that our opponents don't have the installed base of software that we do and, therefore, they can leapfrog and presumably produce even higher software-centric things that we have to deal with.  We need to address this now.

Thank you.

MICHAEL MCQUADE: Thank you.

My name's Michael McQuade.  Along with my colleague, Dr. Richard Murray, I'm one of the co-chairs.  Just a couple of brief comments and then Richard's going to talk a little bit about some of the specific substance.

I would just echo a couple of comments you've heard.

One of the first things we did, even before we determined a plan to take on this study, was to look back at the tens, hundreds of studies around software acquisition that have been done across the government.

And in many ways, I would tell you that, at the end of this process, much of what we have found, you can find in lots of studies that have occurred here over the years.  And in fact, one of the standard refrains we had is, if you look at the 1987 Defense Science Board study on software acquisition and you take the word "ADA" out and replace it with "Agile," you actually have a pretty good study.

What -- what was going to make a difference?  And -- and Eric just talked about this.  The fact is software matters even more now than it has ever mattered.  The capability that we deliver -- while enabled by hardware, and enabled by the amazing men and women who deploy that capability -- is defined by software.  So the problem is more important than ever.

We tried to do this study very differently; we tried to do this the way you do software.  Not just because that's a clever way to do a study, but because that's part of the cultural change that has to happen in the way the department improves its implementation of the -- of the recommendations in this study.  It is an iterative process; software is correctly always done starting small, iterating and moving forward.

The second point I would make is that as you read through the study, please carefully read the sentence that says, "Not all software is the same."  

Because it is very easy to take one of the recommendations in here which is about, for example, how you should acquire off-the-shelf, COTS software, and it's very easy to talk about how that doesn't apply to the embedded code on a -- on a fighter aircraft.  That's precisely the point, precisely the point, that there is different software and it is implemented different ways.  And the recommendations in the study need to think about that as we go forward.

The final thing I would say is that we make a statement early on in the report that says it is, in fact, true that you could likely implement almost everything in this study within the existing processes and authorities at the department.  And in so doing, you would have to be a hero and that would not be the most efficient way to do things in here.

So the recommendations around new software pathways and new colors of money and new processes are fundamentally aimed at recognizing that software is different, and that acquiring, developing and deploying software is not the same as deploying hardware.  And that asking people to optimize a software acquisition system in a hardware framework is not the right way to operate.

So with that, Richard's going to go through a couple things.  I just, because I think we will both do this, I want to -- to recognize and thank the staff members Courtney Barno, Devon Hardy, Sandy O'Dea and Jeff Boleng, who were instrumental in bringing this to conclusion.  And thank the entire department, including Secretary Lord, for the fantastic support that we had.

Richard?

RICHARD MURRAY:  Great, thanks.

So just a couple of things to add to that.

You know, I think as everybody's said, we think that this is the right time for this type of a change within the way that the department actually does its software.  As we went through, there were three main themes as you'll read in the report.

One of them is that speed and cycle time are the most important metrics for managing software.  We have to think about:  how do we get things out there quickly that need to get out there quickly?

A second was that software is made by people and for people.  So digital talent matters, and we have to think about the digital talent within the services, within the contractor base and how to make sure that that is where it needs to be.

And the third, as you've heard already, is that software is different from hardware, and not all software is the same.  And so we need to make sure that the acquisition pathways and the appropriation categories are optimized for software, because software is what defines the capabilities that we have for national security.

So we've tried to go through and think about:  OK, great, those are nice themes but what do you actually have to do about it?  We've articulated four main lines of effort that we think are the most important things to do and, within those, identified just two or three recommendations each. 

There's a top 10 set of recommendations -- there's 16 additional recommendations, but we really focused on the top 10. 

And we’ve put, as Secretary Lord said, an initial, kind of, implementation plan, here’s a -- here's the starting point.  And we look forward to working with the department as it teases out that, and working with Congress as they help enable that.

I think that it is the case that the U.S. private sector has figured out how to do software.  The U.S. Department of Defense, working with Congress, needs to take advantage of that incredible strength that the country has.  And so I think the time is now to do that.  There’s great alignment in terms what's going on.  And so I think we just need to start moving forward and actually go do something.

And I'd just like to echo Michael's comments in thanking the -- the staff and all of the people who've met with us at the Pentagon.  It's been a very interactive process.  We have engaged with lots of people in industry and government all over and so I'd like to thank all of those people -- and in particular, Courtney, Devon, Sandy and Jeff -- for all of their help.

So with that…

STAFF:  Ladies and gentlemen, before we start the questions, out of respect for those who have traveled here today just to talk about this, if we can just keep the question on this.  Ms. Lord is going to do a media engagement next week to talk all things acquisition.

So starting with that, Marcus?

Q:  Scott had his hand up.

STAFF:  Go ahead.

Q:  Scott Maucione with Federal News Network.

Can you explain a little bit more about the -- the color of money and how you're -- you're categorizing it?  The acquisition -- I'm sorry, the budget cycle is one year; Agile software is not a one-year cycle, so...

MS. LORD:  Exactly, exactly.

Q:  … how -- how would that work?

MS. LORD:  So the thought process is right now, if you definitely -- if you do DevOps, what you're going to be doing is development, production and sustainment all at once most of the time. 

Right now, as you know, we have different pools of money that we have to very carefully allocate.  So what we've been talking to the appropriators about is writing in the '20 bill the opportunity to do multiple pilots; where, we would have just one line for software development so we can move back and forth amongst those different stages to give us what I'll call administrative flexibility.

I don't believe our business systems right now reflect how we do software.  So we are asking to have the authority to do some pilots in '20, looking towards some actual new language and new title in '21.

STAFF: Tony?

Q: I wanted to ask you all about the -- the F-35, the -- the program, the biggest program on Earth -- 8,000 -- eight million lines of coding in each airplane. What lessons did you take from your report that, going forward, you need to incorporate into the F-35's so-called block four modifications to prevent, you know, the -- the major issues with blocks two through three (F, ID&C ?)...

MS. LORD: Got it. Perhaps I can lead, and then I'll hand off...

(UNKNOWN): Sure.

MS. LORD: ... to you all for comments, because one of the things that makes this study meaningful, in my opinion, is that the team looked a whole variety of software programs -- ones that have gone well, ones that haven't gone so well so that we can learn from all of them. F-35 is one program that was looked at. I will tell you, as we move forward towards gaining authority to do the pilots this fall, one of the programs that we have targeted specifically is F-35, and primarily ALIS on that, because ALIS had legs into every part. It is thought of as key for sustainability. However, it has tentacles back into development and into operations.

Q: One quick -- quickie -- intellectual property, the (inaudible) recommendation of much -- much ado, are you going to start enforcing that in contracts, going forward? You know industry's resisted giving up their IP.

MS. LORD: What we have done is already worked on standard IP clauses, because we want to make sure our acquisition workforce understands that we need the intellectual property that allows us, for instance, on platforms to spiral in new weapon systems, new center systems, and you have to understand the interfaces in order to be able to do that.

We also want to make sure that we have intellectual property that allows us two, three years in production to compete sustained to make sure we get the best value. So what we are concerned about is making sure that we have rights to the intellectual property that allows us to do those two things. Otherwise right now, it's not as critical, and we're working towards that.

As we evolve our software model, I'm thinking that basically our systems are going to be hardware enabled, but software defined so that we can rapidly spiral in capability. We'll have to look at what that means about intellectual property for source code and so forth.

MR. MCQUADE: I'll -- I'll add one sort of a slightly facetious comment, and Richard has a substantive comment. I'll just use your -- your one comment about eight million lines of code. One lesson we've learned with the F-35 is you should never use lines of code to manage your program in any way, shape or form. In a world where software is developed and deployed by a combination of new technology where computers help write code, software lines of code is precisely a wrong metric for measuring program complexity. It leads to behavior -- if you pay somebody by lines of code, you get more lines of code. Not suggesting that's an F-35 statement, but in general -- and you'll see it in our recommendations -- you should not be measuring lines of code on a program.

MR. MURRAY: And so one of the things we do talk about, of the other metrics you might use, speed and cycle time are the most important metrics, we think, so that's one of the things you want to look at something like F-35 and...

And the other thing I'll just about F-35 and lots of other programs is it's a great example where there are multiple types of software. So indeed, there's some software that is embedded software and flight control software and other things. But there's things like ALIS that really is running on commercial operating systems and on commercial hardware, and you don't have to manage those in the same way, right? And you don't -- the metrics that are there for those might be the same classes of metrics, but with different targets, depending on what they are.

STAFF: (inaudible)

Q: Thank you. Some of the studies that have been done on DOD acquisitions that are software-intensive pointed out that the users were not involved early enough in the development. I think -- I'm sure you've researched that extensively. But can you explain what -- what goes behind a program that doesn't consult the users? I mean, how is that -- how is that even possible that billions of dollars were spent before the users actually got involved in the development?

MS. LORD: So I say we need to earn from the past and focus on the future, and our future focus is to make sure that operators, that users have their voice heard at the beginning of programs, because that comes in in many areas. We have to design for sustainability, for instance. People often don't realize that our platforms' major weapon systems have 70 to 80 cents on the dollar out of the full lifecycle cost on sustainability. So we need to have all those voice heard. And I know that General Selva and the whole JROC process is very, very focused on that.

Q: Have you seen -- have you seen some of those studies like GAO just recently? Were -- were they accurate in -- in that assessment, that...

MR. MURRAY: I'll say, I think what we've seen when we've gone out and -- and looked at some of the programs is that often there's a -- a layer of disconnect, right? That where the requirements are coming from -- and we had talked about the fact that we don't think requirements are the right way to manage everything, but where the requirements are coming from is talking to people who oversee those programs, rather than people who are using that software. If you think about what happens in your everyday life, the way you give feedback on pieces of software, you know, if you like this or not, how many stars, et cetera, you know, how do we do that in an appropriate context? How do we go to the actual users of the software and say, "Hey, we're rolling out this new feature. It's intended to help you do this. Is it or not," right?

MS. LORD: Yeah.

MR. MURRAY: And you can do A/B testing and other things. So how do we get to that style of software follow-up?

MS. LORD: So there are very simple questions we need to ask, and sometimes we make them much more complicated and too focused. It's, what is the problem we are trying to solve? What is the question we're trying to answer? We need to be very fundamental so we get back to basics, and don't overcomplicate this with a lot of details, but really understand what is the core functionality required? And then, what is nice to have?

MR. MCQUADE: The only other thing I think I would say, just very quickly, if -- if you think about the -- the meta-forward model we're trying to predict to, if you think of, ‘I have requirements’, and then I turn it over to somebody to develop, and then you're going to ask the question you've just asked about: Who's involved in writing the requirements? If you think about the way modern software is done, the issue isn't who's involved in writing the requirement. Who from the user community is intimately involved in the entire (inaudible) from beginning to -- not even end, but from beginning all the way through?

So it becomes a very different question. It's not: who was the user who helped make the requirements? It's: who's the user who's part of the Agile DevOps process across the entire lifecycle?

Q: OK. Tony Bertuca, Inside Defense. Thank you for -- for being with us. 

For, Ms. Lord, on the legislative proposal to, sort of, get a color of money designation for software, which areas do you think are best suited for that, for those pilots, for the department to start transitioning into them?

MS. LORD: I think in order to do meaningful -- an overall meaningful pilot study, we have to have a variety of different types of programs because the objective of doing a pilot study is to really learn by doing. That's kind of my mantra. I don't want to sit around conference room tables with hypotheticals; I want to dive in and try to do something in sort of a safe way.

So we will look at some major platforms, also at some business systems so that we can get the depth and breadth. 

Q: And -- and also for you, but for anyone else on the panel, one of the things Mr. Schmidt mentioned was the software environment at DOD is out of the 1990s. 

Are there areas that you can tell us now, that if we can't go back and redo the entire F-35, where can the department put points on the board in this area (inaudible)?

MS. LORD: I think what we have to do is look at programs where we have suboptimal functionality and performance right now and those where we really need step function changes in capability so that we bring the most modern coding to it. 

But I want to emphasize, this isn't just about having software developers use Agile or DevOps, it's about teaching the acquisition workforce how you manage programs like this. So to that end, a large number of the recommendations and things we're moving out on are: how do you develop that capability? 

So we have an enormous effort at Defense Acquisition University to change how we're doing this. We also are rewriting -- as I said earlier, I think -- 5000.02. So it's a full-court press, not just what the technical community, but also with the acquisitions and contracting and sustainment community. 

MR. MURRAY: Just to add a little, you know, I think that's absolutely critical. I think you got it exactly right. You have to have an environment in which people can productively carry out their work. I think that's what programmers want to do, is they want to see their code actually doing something useful. 

So investing in the digital infrastructure that's required to do that, that means cloud computing, that means software stacks, it means all of the things that allow people to quickly develop capability and put it in the hands of users and get feedback on how that's doing. That's what the department's going to have to go do and that's what our recommendation is. 

Line of Effort B, the second one, is exactly on building that digital infrastructure. 

Q: Great, thank you. Hi. Leigh Giangreco of The Capitol Forum. 

The report mentions cloud 115 times but JEDI only once. Why didn't the report more closely examine JEDI as part of the DOD's software strategy? And then why didn't it also weigh in on the debate of single versus multi-provider? 

MS. LORD: I'll give my perspective then allow you. 

JEDI is just one of the cloud efforts within DOD. So I don't think we within DOD single out JEDI from other cloud efforts. Our focus has been more on having, you know, hardened containers in an enclave that can be accessed. We do non-recurring engineering over, and over, and over again in the Department. 

What we're trying to do is build the capability to have a series of hardened containers that can be accessed by any contractor or within DOD. So that we -- if we use our stack, you get -- automatically get your authority to operate, so that's agnostic of what cloud it's in. It's more an approach. 

MR. MCQUADE: I -- I would just echo the same thing, is -- you know, it was not our job, to do a report on the acquisition of a cloud, the department. That's a process that's underway. It will occur and have its own process to get there. 

We simply make the point that if you do not have a cloud infrastructure to be able to operate, many of the things that we talked about in this report simply won't happen. 

MS. LORD: And remember, one of our objectives in what we're doing is to have secure and resilient ecosystem in which to develop software. And this whole methodology of going to the cloud should help us enormously there, especially when we talk about small companies. 

A large amount of our innovation comes from small companies. In this environment where cyber-security is absolutely paramount, we cannot always find a way to have small companies be able to afford to harden their systems. 

So part of what we're looking at within the department is to provide that secure environment in which to develop software. So we want to provide that environment in a cloud where that is needed by a company. 

STAFF: Go ahead, Mike. 

Q: Just -- sorry. Can I just ask you to clarify, if you can put in, like, layperson's terms? When you say a hardened containers, a series of that, what -- what that means, what it would look like? 

MS. LORD: So what we want to do is have a standard number of software tools or ways to do different things. Break those into the lowest -- the smallest buckets we can. And then make them so that they cannot be hacked and so that they also are updating automatically with new feeds to that software. 

And the idea is to go in and get all the different little buckets you need. But if they're constructed in a very disciplined way, once you go and you build your code -- by virtue of the fact that you have used these -- you automatically get your authority to operate because we know it meets our security criteria. 

Q: OK. That's helpful. Thanks.

Q: Mike Stone from Reuters. Thanks for doing this. 

So just so I understand, we're starting off with -- is it rainbow money in a wallet that will be used for pilots? What's the size of that initial authority that you're going to Congress with? And then where do you see it growing, over how many years?

First part of the question, you talked about...

MS. LORD: OK. I'm going to lose track here soon. So let me answer...

Q: ... Sure.

MS. LORD: ... that part of it. We are not asking for any money for these first pilots. We are asking to look across different lines, and pulled money, and use it. So this is more administrative flexibility versus asking for money. 

Q: And you talked about small businesses. Oftentimes, smaller businesses that do DevOps currently are layered by an ops fixed contractor, who takes a fee. How are you going to reach out to those companies -- very small contract size, much more frequent contracts for these folks -- to come in and avoid, say, a 15 percent charge? That someone...

MS. LORD: So you're really getting at the crux of one of our objectives, which is to de-couple software and hardware and get the best of breed for both. So what we have to do is be able to structure our contracts, our programs to be able to do that. We also have to make it easy for small businesses to come and work with us. 

And we would be successful if industry thought of Acquisition and Sustainment as their big helpdesk. They call in to us, our Industrial Policy group reaches out, explains how to work with us. We partner very, very closely with numerous industry organizations -- PSC, NDIA, AIA to let them know what we have for resources. 

So part of the implementation of this study is a communications campaign to get out there and reach industry, especially small business. We actually have a new director of industrial policy, Jen Santos, who will be starting on June 3rd. We have an opening to run our small business office. We are right now interviewing people for that. That will be a key objective of that small business office, is to reach out to small businesses who want to develop software for us.

STAFF: We can go right over here. (inaudible)

Q: Thanks. So I wanted to circle back to what Dr. Schmidt said earlier, which was the initial intent with the board was not to do reports And so we now have a report here. But the board was created by the last administration and inherited by this one. I wanted to ask, how has the work of the board, how have the processes and intent of the board changed with the change in administration? What are you looking at doing differently now, or is it exactly the same?

MR. SCHMIDT: I can say it seems completely the same. What do you guys...

MR. MURRAY: Yes, I agree.

MR. SCHMIDT: our -- all of these other machinations have been largely transparent to us. I mean, we made an initial set of end recommendations now, roughly 16. The DOD is busy implementing a whole bunch of the recommendations and we're told that they're moving quickly. Now, in my world it seems a little slow, but -- but -- but...

MS. LORD: We're catching up. We're catching up.

MR. SCHMIDT: But -- but you know, frankly, we -- it -- it takes a leader like Ms. Lord to actually drive this stuff, and so we have it in her, so now we have the software stuff actually happening, right? As we go through in the -- in the report, people have made these kind of software reports for 30 years. Well now, it's actually happened because she's figured out she has to have it done and she cares about it, and she's going to work with Congress to make it happen.

STAFF: Travis?

Q: Hey, Travis Tritten, Bloomberg. Thanks for being here. Appreciate it.

The report says that the GS employment system and the military structure, as well, favors time over talent, and that that simply doesn't work when you're talking about software. So I'm wondering how you get at that problem. How do you get around that issue?

MR. SCHMIDT: I'll tell you, we have lots of stories. We were visiting in some deep, deep, window -- windowless room somewhere here, and we had a presentation on something involving Russia from a very young military officer, and we -- and we thought he was incredibly knowledgeable about cyber stuff. And he seemed unhappy, and so I said, like, "What's going on?" And he said, "Well, I'm getting transferred to some other division next year." "How long have you been doing this?" "One year."

So if your basic model in the military is that you get to do cyber and software for one year, and then you go work on, you know, marketing, sales, or whatever it is he's going to do next, you don't understand software. Software is a -- is a very special thing. It's like being a doctor or being a -- you know, that kind of stuff. And we've said repeatedly that the military has to have a technical track for career people in the military and also civilians at the same level as a technical company. 

STAFF: Last question. Jeff?

Q: When it comes to lessons learned, would it be fair to say that ALIS represents everything that is wrong about how the DOD approaches software? And how do you avoid having another ALIS in the future?

MS. LORD: I never say (inaudible) always. 

(CROSSTALK)

I think ALIS embodies a lot of requirements changing over time, as well as moving implementation. I think we can learn a lot of lessons from ALIS. I will say that the F-35 is the premier fighter aircraft today, and that it is performing very, very well. 

MS. LORD: There are things that can be improved, and we will improve them, and we do take a lot of lessons from ALIS. But we are committed, I am committed to making ALIS a premier system. 

Q: And you had mentioned that it's not good metric to pay by the line of code. Is -- are companies taking advantage of the Defense Department by adding in lines of code that simply don't need to be there...

MR. SCHMIDT: No.

Q: ... so they can build? 

MR. SCHMIDT: No. No, it -- my -- my point is a different point. My point is that that is not the way modern software is done. So imagine that you did -- we -- in the report we talk about DevSecOps which is a cycle, and imagine that you had a software team and their job was to build today what ALIS does. 

They would operate in a way that's quite different, according to this report, than the way the F-35 program was conceived in 2001. So the fact of the matter is software is different. And the changes that we're proposing, that I think we're going to get through, are a material change in the way software like ALIS will be done. And that was not available to the contractors, and the procurement and all the various other people involved with it, until now. 

STAFF: Ma'am, do you have any closing remarks? 

MS. LORD: I just want to thank the Defense Innovation Board for their all-in approach. This has been a contact sport, which is great. We've had a lot of cross-functional teams and I'm particularly excited that they have committed to working the implementation for the next 12 months. So thank you all very much. 

MR. SCHMIDT: Thank you. 

MR. MURRAY: Thank you.

MR. MCQUADE: Thank you.

MR. SCHMIDT: Thank you, guys.