Second Keynote Dan Geer

Transcript

at this time I'd like to ask Stefano sanara our international conference chair to the podium hello thanks Andrea for standing up to the podium even vihar Karekin dishes but really I'm sure we really appreciate you talking to all of our members and try your best yesterday evening now at this time I'm really delighted and to introduce our keynote speaker for today this this year we have been really fortunate to have two of the luminaries of the profession talk to us today we are going to listen to them gear then is a security researcher describes himself as a security researcher with a quantitative band as a security researcher myself I can certainly appreciate all of its brighting and is one of the few people that I really tend to listen to very carefully whenever he chooses to speak or write from MIT in the 1980s where is group produced something like Kerberos that I think everybody knows through a number of startups and today is the season of ink you tell which is the strategic investment arm of the US intelligence community so ladies and gentlemen please join me in welcoming to the stage than gear good morning thank you for that and it's a pleasure to be here I hope you take that as a genuine statement and not a cliche I'm not going to do this with slides I'm going to do it with prepared remarks and they will be available in text form to you through the office immediately after and sort of like that old Jay Leno commercial eat all your like will make more in this case do as you will with it as far as I'm concerned any time you speak in public you've in so many words given up the right to to copyright or anything else this is entirely for you and I won't take questions from up here but I will hang around for as long as people want to talk afterwards those of you who at the end of the talk wish to escape are welcome to do so then in that seat out there so I know how that goes alright so my topic for today is intelligence and it's a daunting topic that of intelligence its history is long but its greatest successes are not necessarily chronicled in objective ways and ditto its failures its scope ranges from nation-states down to the personal though the character what constitutes intelligence dramatically changes with scale of course the word intelligence as we will use it here today is that of an activity not a description of a human being however note that the word intelligence derives from the latin verb in telugu to comprehend or to perceive and so one might say that intelligence as a term and psychology is not all that different than intelligence as a term and national affairs i would therefore suggest that intelligence is that which enables decision-making and in turn which improves outcomes they're still passing vague but intelligence as a contributor to two decision making is the idea that i will hold throughout this talk there are other aspects of decision-making principle among them timeliness and we'll come to that in due course as some of you know with two others we started a mailing list in a workshop series on security metrics that was unbelievably ten years ago and for me at least the entire point of security metrics both then and now is that of decision support I respect science and those who chronicled a natural world but chasing down numbers that let me make better decisions is what security metrics and in the greater context intelligence is all about here are my quote Edward Tufte the well-known statistical graphics guru that quote if your numbers are boring you've got the wrong numbers that allows me to state his comment in the contrapositive namely that if you have the right numbers you're not going to be bored there's a question here of sorts and namely whether we have decisions to support for which intelligence can usefully play a rule the straightforward answer is of course it can in fact intelligence is essential the somewhat less straightforward answer is it depends on the intelligence to the extent that one can assess one's own biases minor that while numbers are endlessly fascinating they are not to be sought for their own sake but rather for that purpose or decision support medicine where I worked for 15 years has a term of art that encapsulates the idea the term is no therapeutic difference meaning that if a blood test or a scan or some other imaging of what what you have might improve the subtle precision of a subtle diagnosis it if the therapy would nevertheless remain the same then the pursuit a further diagnostic precision is not medicine but something else in other words and as we say on the security metrics mailing list I'm a measurer not a modeler but let me be clear about one thing that may make cyber security different than everything else and that is that we have sentient opponents the physicist does not the chemist does not not even the economist as sentian opponents we do what puzzles we have to solve are not drawn from some generally diminishing store of unsolved puzzles nor could our theories completely explain all observable fact thus reducing our worries and our work to engineering alone there is something different about a search for truth when there isn't any or at least there isn't any that lasts long enough to exhaustively explore there's something different when what we can detect and from and from which we can then infer is even partly under the control of people who have crossed purposes to ours the word intelligence as we're using it here certainly has a military ring to it as I've said before in the 1990s the commercial sector caught up with the mill sector and the application of cryptography not as design by this application we're now passing the knee of the curve for the commercial sector to catch up to the mill sector and traffic analysis on the commercial side we may be also catching up and the use of disinformation though that is inherently harder to ascertain harder still is ascertaining if the commercial sector is engaged in covert action but there is little doubt that the commercial sector has now deployed more listening points than has the mill sector what the commercial sector is however rediscovering is that it is easier to drown in information than it is to profit from it it's one of those lessons that can only be learned the hard way there are in other words parallels between cybersecurity and the classic military intelligence functions insofar as predicting the future as a strong role to play and preparing your defenses for probable attacks as Dave I tell has repeatedly pointed out the hardest part of crafting good attack tools is testing them before deployment knowing what your tool will find and and how to cope with that is surely harder than finding an exploitable flaw in the first place this too may grow in importance if the rigor of testing causes attackers to use some portion of the Internet at large as their test platform rather than whatever rig that can afford to set up in their own laboratories if that is the case then full-scale traffic logs become an independent indispensable intelligence tool in so far as when an attack happens and it appears to be de novo those with full-scale traffic logs may be in a position to answer the question how long has this been going on the company netwitness which is now part of emc is one player that comes to mind in this regard and there of course others this idea of looking backwards for evidence that you didn't previously know enough to look for does certainly have intelligence value of both for the nation-state and for the enterprise the idea of looking forward with what people are willing to reveal an open source has even more value as the company recorded future has demonstrated to my satisfaction when one is doing science one collects data and tells everyone how that collection was done so that where necessary your findings can be refuted by others as you may have read science is in a crisis of sorts in this very regard some notable fraction of all scientific papers cannot be reproduced as one analysis of this building crisis said and I quote reproducibility is a defining feature of science but the extent to which it characterizes current research is unknown replication success is better predicted by the strength original evidence than by characteristics of the original or the replication teams that's an important observation the trustworthiness of an inference is proportional to its reproducibility and its reproducibility is proportional to the strength of the evidence collected but is that true only for science one might go so far as to say that the importance of the decision to be made must be reflected in the quality of the information that is collected or maybe not there's certainly a lot of hard decisions decisions to make now for which good data is not readily available in any sense you surely know some of this but speaking from a statisticians point of view which incidentally is what I was originally trained as you would prefer that your measurements whatever they are be both accurate and precise when that is not possible there are trade-offs to be made if your measurement instrument misses a lot then it may be that you cannot believe the numbers it produces at least per se but even a poor quality measurement will yield trend lines even if the actual value is given are erroneous so long as those errors are not closely correlated with the value is being measured then the shape of the trend line will be correct spoken of differently a situation where you have sampling but have no way to tell what your sampling fraction is will steal yield trend line data so long as your sampling fraction is stable in any case I would argue that absolute measures are not the goal in cyber security that relative measures are sufficient for the simple fact that relative measures like such and such a risk is getting worse are sufficient for decision support this has parallels in the physical world no police department will ever know how much heroin is for sale but they can tell the price and a rising or falling price is sufficient for decision making about what to do and whether what was done has had a positive effect over a small number of years the term data science has become commonplace it seems first to have been used over fifty years ago but the current usage stems most directly from a 1997 lecture by Geoffrey woo with the title the statistics equal data science will characterize statistical work as a trilogy of data collection data modeling modeling and decision making in his lectures conclusion he initiated the modern use of the term data science and he advocated that statistics be renamed data science and statisticians be renamed data scientists those semantics seem to add little clarity to what the collection modeling and use of data provide but argument over terminology is a hallmark of how a science proceeds which brings us to ask whether cybersecurity is or could become a science a few months ago I've tried to round up some ideas on that very question taking as my god TS cones are seminal work the structure of scientific revolutions the question of whether cybersecurity is yet a science is a hard one I'm sorely tempted to answer the question is cybersecurity of science with getting closer but not yet to say in other words that we are in a prescience stage with several others I'm one of the expert reviewers for the national security agency's annual science of security competition and award quoting its rationale the competition was established to recognize the current security paper that best reflects the conduct of good science in the work described science of securities abroad Enterprise involving both theoretical and empirical world while there can be only one best paper anyone paper cannot span that full breadth nonetheless the field is broad and all facets are encouraged and needed the common denominator across the variety approaches is solid methodology and effective communication so those aspects of the papers are strong factors and our decision making so that would be something that you might want to look at is the science of security award every year the next one we announced in November and of course if you particularly fond of a big given paper nominating that for next year would be helpful in fact papers are nominated for consideration and I encourage you to do the exactly that but I'm also here to report that amongst the members of the reviewers reviewing committee our views of what constitutes a quote science of security unquote very rather a lot some of us would prioritize purpose are agreeing with Charles Darwin that all observation must be for or against some view if it is to be of any use some of us view aspects of methodology is paramount especially as hinted above reproducibility and the clarity of communication on which it depends some of us are ever on the lookout for what a physicist would call a unifying field theory some of us insist on the classic process of hypothesis generation followed by designed experiments we vary and I take that to be a vote of sorts of whether cybersecurity is yet a science but Darwin's remark that all observation must be for or against some purpose is not quite right at least not quite right for us here it is not so simple as Wu's data collection data modeling and decision-making when you collect data and with it build a model your goal matters if your purpose in building a model is to come to a definitive conclusion about causality about how nature works then you're saying that the inputs to your model and the coefficients that calibrate their influence within your model are what matters parsimony in the sense of Occam's razor is your judge aura saint-exupéry described you know that you have achieved perfection in design not when you have nothing more to add but when you have nothing more to take away by contrast when your purpose in building a model is to enable control of some purpose or other then you will not mind if your input variables are correlated or redundant their correlation and the redundancy are not an issue if your goal is to direct action rather than to explain causality in some circumstances you can do both that is you can both explain causality and enable control and those situations it is your models ability to do prediction that both satisfies the reader that you have captured the causal relationship and that the models predictions can be operationalized irrespective of underlying truth a global of understanding causality and it's full elegance leads to F equals MA or e equals MC squared or the like a goal of control leads to econometrics with thousands of input variables each of whose individual contribution is neither clear and irrelevant that may all seem harsh but it is not meant that way at all consider anomaly detection and its role and current cyber security products anomaly detection presumes something about distributions of detectable events namely that within a selected interval anything outside some bounding box is worth investigation it is not concerned with causality it is concerned with control irrespective of causality that is a coherent site strategy to be sure and has some side effects or consider big data and deep learning even if Moore's law remains forever valid there will never be enough computing hence data-driven algorithms must favor efficiency above all else yet the more efficient the algorithm the less interrogate able it is a term of art from Michael Osborn a professor at Oxford that is to say the more optimized the algorithm is the harder it is to know what the algorithm is really doing that was the exact theme of a workshop held by Morgan Stanley and the Santa Fe Institute last fall titled or optimality and efficiency the enemies of robustness and resilience all the speakers answered yes in one form or another and there's a feedback loop here the more desirable some particular automation is the more data it is given the more data it is given the more data utilization efficiency matters the more stated utilization efficiency matters the more its algorithms will evolve to opaque operation above some threshold of dependence on such an algorithm in practice there can be no going back as such preserving algorithm interrogate ability despite efficiency seeking driven automation is the research grade problem that is now on the table if science does not pick this up when Larry Lessig's characterization of code as law is fulfilled a couple of law professors in fact have seized on that very idea and suggested that price-fixing collusion amongst robot traders will be harder to detect inclusion amongst people I am involved in a few of the many information sharing initiatives that seemed to be everywhere at once and one and speaking in the context of control this comment was made some of your data suppliers may not want to send you data if you display it as soon as you get it as that may be used by bad people to notice that they have been detected or to game the detection systems sure you can obfuscate the data but that's a lot of work to gain a few minutes or hours of statistics now that writer was emphasizing that we are operationally likely to choose control over basic research and that we have real opponents but the point is this if we choose control as the purpose of our efforts then we will have to let causality become harder to see because our models will submerge any causal relationships in a thicket of confounding if instead we focus on causality the very things that we need to measure become harder to get if for no other reason our centium opponents will make that so as I said earlier I'm for measurement as decision support I am in the control camp not the causality camp at the same time I do want to be able to ask some algorithm why did you do that and get a meaningful answer overall having both control and interrogate ability is a difficult problem to say the least what I want is to predict the future I want it for reasons that are no doubt emotionally clear but I also want it because of my own definition of security which is this a state of security is the absence of unmitigated surprise as always in cybersecurity we're now talking trade-offs one of those is in deciding how many failures is the right number of failures it can't be unbounded upward that's obvious it can't be zero either as zero quite likely means that you're overspending and in any case learning from failure is especially crisp as francis bacon said truth emerges more readily from error than from confusion defining a state of securities the absence of unmitigated will surprise mirrors what I know to call the availability calculus namely that we can get a hundred percent availability by driving the time between failures to infinity or by driving the time to repair 20 I am searching for prediction because I want to drive to infinity the time between failures for which I have no mitigations and to drive 20 the time to repair failures for which I do that is where the focus on control leads you or at least I'm so think at this time but one might ask why why prediction why speculate the late michael crichton gave a magnificent essay with that exact title in april two thousand two boiled down Crichton simply said that no one knows the future and those who pretend to do so are self-serving delusional or something equivalently uncomplicated achter say he notes how big the prediction industry has become mind you this is 2002 singling out media especially and he reminds us that the track record for sweeping predictions is pretty poor he coins a clinical term and I might as well quote his text where he does so I'm now reading Crichton media carries with it a credibility that is totally undeserved you have all experienced this and what I call the murray gell-mann amnesia effect i refer to it by this name because i once managed to discuss it with murray gell-mann and by dropping a famous name i imply greater importance to myself into the effect and it otherwise would briefly stated the Gelman amnesia effect as it follows you open the newspaper to an article on some subject you know well and Murray's case physics and mine show business you read the article and see the journalist has absolutely no understanding of either the facts of the issues often the articles so wrong it actually presents the story backward reversing cause and effect I called these wet streets cause rain stories papers follow up in any case you read with exasperation or amusement the multiple errors in a story and in you turn the page to national or international affairs and read it as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read you turn the page and you forget what you know that is the Gelman amnesia effect I point out that it does not operate in other arenas of life an ordinary life as someone consistently exaggerates relies to you you soon discount everything they have to say in court there is a legal doctrine of false in uno false and omnibus which means untruthful in one part untruthful and all but when it comes to media we believe against evidence that is probably worth our time to read other parts of the paper when in fact it almost certainly is not the only possible explanation for this is M neva that's the end of gilman of Crichton's remarks everyone here knows exactly what Crichton was talking about or was 13 years ago what is written about cybersecurity for the general audience is sadly often counterfactual or count illogical unfortunately what is written for specific audiences including legislators and regulatory agencies can also be counterfactual or counter logical and all of this finds an audience because of an actual need that I argue is acutely important for cybersecurity we need to protect predict is the future if our tools are to intersect our problems on target and in time there's a theme here the fast-moving nature and yes the ability of the cyber security regime are such that we're occasionally possible to make useful predictions we would be much better off much better able to accomplish our security plans while those plans were still meaningful at the same time and especially in cyber security no one can predict the future we desperately need prediction we know it it is near impossible to do and increasingly so at the same time I am myself entirely guilty of trying to do prediction and cyber security I give speeches on this precisely I'm working on a personal project right now whose only point is prediction with a quant colleague of long run another I work on the periphery of the intelligence community and the intelligence community is entirely about prediction constantly speculating on what is our actual position and what is our actual velocity if you're very job is security in any sense then you want all the prediction you can get yet at the same time surprises happen if he were still with us Crichton would remind us that the problem with speculation is that it piggybacks on the Gelman effect of unwarranted credibility making the speculation look more useful than it is one can argue that compliance is a predictive exercise based on the idea that if you do this thing then you can approach the future with less to fear i buy that train of thought and do so wholeheartedly but what if the rules to be complied with cannot keep keep up with the rate of change if they cannot then whatever the prediction of outcome that compliance promises is prediction made relative to conditions that no longer hold that can't be good that can't be useful unpredictably unpredictability is so true in cyber security that we have a special name for when prediction fails zero day we accept that a genuine zero day as an attack that no one could have seen coming we so very often imply that failing to handle that zero day is blameless since after all it was not predicted yet every time lurid zero-day shows up I find myself asking could I have predicted that how Bruce Schneier ask a question is simple to state a question we have yet to answer a question we're getting the answer right will change things our vulnerabilities sparse or dense if and only of vulnerabilities are sparse does it actually make sense to allocate the effort to find them or reward those or do or to set up appt Iraq I for sharing information about those vulnerabilities the vulnerabilities are dense then treasure should not go to finding them but to making systems resilient to them this speaks directly to security is the absence of unmitigated or surprise does it not irie asks nars question in various settings and I get strong and I mean strong opinions over the full range of dense to sparse smart knowledgeable people say too dense to measure while smart knowledgeable people say too sparse to measure that's not a trick question it's a steering question like no other in a similar vein it seems to me that the broad in general we are near a fork in the road a fork where one road is that of generating provably defect-free code followed by long-term brutally rigorous change control while the other road is that a moving target defence rapid release DevOps etc these alternatives seen both antithetical get promising and both are fed by real scientists making real progress but after having chosen one road switching to the other at some later time would seem to be infeasible this speaks again to the availability calculus do we maximize the mean time between failures or do we minimize the mean time to repair we cannot do both nor therefore should we try I will predict that as automation and artificial intelligence proceeds we will find that vulnerabilities are in fact dense there is a recent paper that is very much worth your time to read global cyberspace is safer than you think real trends in cyber crime written by Eric Jardine and released by the Chatham House this past July it's message is exactly that given by its title that cyberspace is getting better not getting worse that cyberspace is getting more safe not getting more dangerous this argument the argument for that message is that to think cyberspace is ever worse ever more dangerous comes from failing to properly normalize whatever measures of safety heretofore been paying attention to it is only fair to quote the front matter of his paper direct directly information technology security firms such as norton symantec and kaspersky lab's publish yearly reports that generally show the security of cyberspace to be poor and often getting worse this paper argues that the level of security in cyberspace is actually far better than the picture described by media accounts and IT security reports current currently numbers on the occurrence of cyber crime are almost always depicted in either absolute a thousand attacks a year or as over as year-over-year percentage change fifty percent more attacks in 2014 in 2013 to get an accurate picture of the security of cyberspace cyber crime statistics need to be expressed as a proportion of the growing size of the internet similar to the routine practice of expressing crime as a proportion of population such as 15 murders per thousand per year in particular the absolute numbers tend to lead to one of three misrepresentations first the absolute numbers say things are getting worse while the normalized numbers show the procedure situation is improving second both numbers show that things are improving but the normalized numbers show that things are getting better at a faster rate and third both numbers show that things are getting worse but the normalized numbers indicate that the situation is deteriorating much more slowly than the absolute overall global cyber space is actually far safer than commonly thought that was the front matter of his paper and he goes on from there in short Jardine is saying that the denominator matters that reporter reporting counts of anything is poorer decision support and reporting rate than reporting rates and proportions that counts of events per unit time will and must mislead is incorrect to talk about how much mayhem there is without talking about how much opportunity for mayhem there is jardines line of critique is entirely straightforward and cyberspace is not the only place where such arguments about the validity of inference are taking place as a prominent example consider Steven Pinker's 2012 book the better angels of our nature while violet why violence has declined in a synopsis in The Wall Street Journal he wrote we tend to estimate the probability of an event from the ease with which we can recall examples and scenes of carnage are more likely to be beamed into our homes and burned into our memories than footage of people dying in their beds of old age there will always be enough violent deaths to fill the evening news so people's impressions of violence will be disconnected from their actual likelihood this is again an argument for looking at rates and proportions rather than counts but in a direct cross nacim nicholas taleb responded with a paper on the super additivity and estimation biases of quantile contributions and which his argument is that when a distribution is fat-tailed estimations of parameters based on historical experience will inevitably mislead a quote from talib now when I finished writing the Black Swan in 2006 I was confronted with ideas of quote Great Moderation unquote by people who did not realize that the process was getting fatter and fatter tails from operational and financial leverage complexity and our dependence and things like that meaning fewer but deeper departures from the mean the fact that nuclear bombs explode less often than regular shells does not make them safer needless to say that would the arrival of the events of 2008 I did not have to explain myself too much nevertheless people in economics are still using the methods that led to the Great Moderation narrative and burn a key the protagonist of the theory has had his mandate reviewed renewed and to highlight a central point we are undergoing a switch between continuous low-grade volatility to the process moving by jumps with less and less variation outside of those jumps now having read it you may find Talib's paper difficult but he is speaking to our interest in cybersecurity are we getting worse or are we getting better is there anything we are currently measuring that is leading us to conclude that we are doing the right things as inferred from measurements of what we believe to be the outcomes are our inferences confounded with little understood assumptions about thin tails and Gaussian distributions when we're actually in a fat-tailed power-law situation are we moving into a world where as Talib's just we are switching from continuous low-grade volatility to less frequent but much larger jump change in the state of our world so a cyber secure the cyber security space getting better or getting worse jardine asks us to normalize how many events did occur to the size of how many events could have occurred not how many did occur in an interval of unit time he is correct that possible that the possible event space is expanding dramatically accelerating and in suspension by all accounts part of that is network extent which I've estimated is having a 35-percent compound annual growth rate part of that is the question of attack surface per se in any case jardine is right that when we count events we are misleading ourselves as to whether we are getting better or getting worse but just changing the divisor alone really make the correction that we need there is a power law here to be sure wikipedias concise reminder is that quote power laws have a well-defined mean only if the exponent exceeds two and have finite variance only when the exponent exceeds three most identified power laws in nature have exponents such that the mean is well defined but the variance is not implying that they are capable of Black Swan behavior end of quote that my friends is our situation cyberspace does not have a well-defined variance for what can go wrong and hence cyberspace is unarguably capable of Black Swan behavior the late Elroy dimson is quoted by peter bernstein famously suggested that the definition of risk is that more things can happen than will and our rate of growth and interdependence is absolutely making the number of things that can happen much larger unfortunately complexity prevents us from counting the number of things that can happen hence jardines argument that we divide the number of things that did happen by the number of things that could have happened is correct in spirit but would be irrelevant if our estimate of the number of things that could have happened were to be wrong however if the denominator is a number of things that could have happened and we severely underestimate that denominator doesn't make that make the news even better talib says no and emphatically the fat tails of power law distributions enlarge the variance of our estimates leading to less frequent but more severe failures the best one could say is that most days will be better and better but some will be worse than ever everything with a power law underneath has that property think earthquakes and whether one is overdue in California and cyber spaces interconnectivity and interdependence are inherently power law phenomenon in an article in the San Francisco Chronicle Thomas lean recounted how quote i found myself at a dinner in a fancy Menlo Park Hotel to discuss cybersecurity with the executives of top Silicon Valley firms the mood was decidedly grim a devastating cyberattack is likely to occur in the neck five years set atop HP exact companies are nowhere near prepared for it neither are the feds there are plenty of comparisons to hurricanes and earthquakes a slow-moving train record one executive said there is a kind of collective cognitive dissonance in Americans thinking about tech will eagerly pursue new innovations like the Internet of Things and electronic health records even as we're increasingly aware of how vulnerable such technology makes us to terrorists and criminals what struck me about the dinner attended by executives from hewlett-packard Cloudera paypal and several others along with academics and investors was the naked pessimism expressed by those in the room no one even tried to put a happy face on the situation end of Thomas League but I ask you are those executives investors and academics getting the right answer for the wrong reasons are Jardine and or Pinker getting the wrong answer for the right reasons is it a truism that when whisk cannot be estimated it will be underestimated how will how do we tell if we're getting better or getting worse and how can we explain this to citizens to regulators and to reinsure errs is talib right that fat tail distributions and asymmetry or where the risk lies and which is more that the apparent suppression of small failures is balanced by yet to be observed Black Swan excursions saying it again if the distributions of events that we care about in cybersecurity are describable not as Gaussian but is power logs then the best one could say is that most days will be better and better and some will be worse than ever everything with the power law underneath has that property and I submit that cyberspace is enter conductivity and interdependence are inherently power law phenomenon a fat tail setting inherently the resist prediction but for that very reason prediction is ever more compelling to pursue so we get published predictions lots of them many of them hedge their bets by phrasing their predictions as a question but that only invokes what I know to call bed urges law of headlines which is that any headline that ends in a question mark can be answered by the word no it's a quandary fast change means tool sets for protection always trail the need unless the need can be forecast fast change makes forecasts hard if that fast change is one of adding mechanisms not just scale to the equation we've got both scale such as an Internet of Things with a thirty five percent annual growth rate and meet mechanism such as afterthought interconnection of sundry gizmos each with new interfaces but to be deadly serious about cybersecurity requires that either we damp down the rate of change slowing it enough to give prediction operational validity or we purposefully increase unpredictability so that the opposition's targeting exercise grows too hard for them to do if the former we give up various forms of progress and the latter we give up various sorts of freedoms as it would then be the machines in charge not us but look at that I can't even talk about prediction without making a prediction why does this matter there is no argument whatsoever that the proliferation of devices and information is empowering it is categorically true not to mention obvious that technology today is far more democratically available than it was yesterday and less than it will be tomorrow 3d printing the whole maker community do-it-yourself biology micro drones search home automation constant contact with ever whom you choose to remain in constant contact with instrumentation of every stripe and caliber the steady migration of military technology to general government use sense to the rich sense to the lumpen proletariat these are all examples of democratizing technology this is perhaps our last fundamental trade-off before the singularity occurs do we as a society want the comfort and convenience of increasingly technologic and visible digital integration enough to pay for those benefits with the liberties that must be given up to be protected from the downsides of that integration if risk is that more things can happen than will then what is the ratio of things that can now happen that are good to the ratio to the number of things that can now happen that are bad is the good fraction growing faster than the bad fraction or the other way around is there a threshold of interdependence beyond which good or bad overwhelmingly dominate now that we need cyber security protections to the degree that we do to whom does the responsibility devolved if the very worst laws are those that are unenforceable what would we hope our lawmakers say about technologies that are not yet critical but might soon be do we forbid becoming critically dependent on them when it is the sheer magnitude of their adoption that makes them critically essential how do we think about intelligence in this setting growth in personal power has meant that here too for military needs like traffic analysis have become common needs one then asks whether when entities under attack be the enterprise's carriers or individuals garner enough provenance to engage in strike back should they as a righteous societal good we by law require that persons with disabilities not be prevented from the fullest possible participation in our society can we find the wisdom to do something equivalent for those who cannot or to the point will not accept the plethora of technologies that are redefining what full participation in society means can we preserve those options simultaneously and if so is that the surest way to prevent common mode failure the need for what we have here 24 called cyber security is now so varied that is no longer a single field but many there are over 800 perhaps over a thousand cyber security startups in some paid stage of the funding game a fair fraction of them spin outs from highly focused university research projects generalist such as myself cannot be replaced there's too much for the novitiate to learn the core knowledge basis reached the point where new recruits can no longer hope to someday become competent generalists serial specialization is the only broad option open to them as I say often cyber security is perhaps the most difficult intellectual profession on the planet Ray Kurzweil is beyond all doubt correct that within the career lifetime of nearly everyone in this room algorithms will be smarter than we are and they will therefore be called upon to do what we cannot and that is to protect us from other algorithms and to ask no prediction protect asking permission and so doing do we like Ulysses like ourselves to the mast or do we as some say would say relax and enjoy the inevitable what would we have science a do what are the possible futures we will tolerate what horses do we not want let out of the barn where do we put our intelligence budget I've asked more questions and given answers it's the best I can do there's never enough time thank you for yours you