Qualities of an effective CISO (2)

‪SANS.EDU‬‪ is updating our strategic plan. In July, I asked my Linkedin network for help in documenting what knowledge, skills and abilities an effective CISO would need to have. Then I asked the GIAC Advisory Board. The list below is my best effort to create a synthesis of the replies. Please note, the rank order does not imply importance, I tried to group similar things together, but it will be a future step to rank these in importance.‬

UPDATE: 8/14/16 We are now trying to rank these values using the ANSI/ISO/IEC 17024 Personnel Certification Accreditation Program approach, to prioritize them based of the ICF values as shown:

Important: these elements are needed for the job and are considered core knowledge.

Critical: failure to understand or execute properly on the elements could lead to harm.

Frequent: these elements are used in the job on a frequent basis.

Selecting a 1 for any of the ICFs mean that is NOT highly valued, a 5 for any ICF means it is very highly valued.

If you are willing to help with this next step please take the survey here:

https://www.surveymonkey.com/r/CISO_Survey

Here are the elements defined and refined to date:
1. Broad knowledge of cybersecurity principles, ranging from technical to human to physical.

2. Deep knowledge of the cyber threats we face today, and tomorrow.  A CISO must understand the motivation behind malicious actors, and knowledge of the techniques, tactics and procedures used can help us better defend, detect, isolate, and recover from the inevitable. 

3. Ability to understand the balance between risk and security, and how to integrate this into a given organization. Able to facilitate discussions about risk.

4. Expert experience in at least one cybersecurity discipline

5. Capabilities focused, not vendor/tool focused  

6. Incredible organizational ability, keep people on task and focused in order to build, design, deliver, and expand the information security program.

7. Ability to prioritize and triage incident response and vulnerability remediation in a calm balanced manner.

8. Ability to effectively communicate technical information to non-technical audiences. 

9. The ability to tie business context/concept to data protection and technical components within the IT and InfoSec space. Doesn't mean they need to be an expert; means that they need to be able to connect these dots for other business leaders. 

10. Liaison between technology and the business, collaboration focused.

11. Able to build relationships and partner with Directors and Board members, particularly outside of IT

12. Able to integrate with the company's mission, and with other division's agendas. Understands the business.

13. Ability to produce and manage to a budget.

14. Clear understanding of which battles to fight, ability to prioritize when there is always more work to be done than time or resources.

15. Design, build, manage an effective security awareness

16. Ability to grow the team as the organization grows.

17. Ability to attract and retain top tier technical talent as well as develop less experienced team members

18. Relationships (inside the org and out), (think Tipping Point connector)

19. Mentor and communicate (up and down)

20. Build and Take care of the team you are privileged to lead

21. Creative thinking, able to apply adaptive strategic and tactical thinking.

22. Works well under pressure

23. Life long learner

24. Resume demonstrates loyalty to organization, a good CISO does not job hop

25. Humility, (admits that he or she doesn't know everything)

26. A sense of humor.

27. Masters degree preferred, especially MBA

Parking lot

CISO reports to: CSO, CEO, CIO?

CISO: Build relationships

In 2016 for a SANS Technology Institute project to insure we were positioning graduates for success, I ran a series of polls on Linkedin and also with the GIAC Advisory Board on the characteristics of a successful CISO. We then ranked those by ICF values, (Important, Critical, Frequent). The highest scoring value was building relationships. This post is a survey of my chosen tips from the top ten Google returns on the subject. My request, is that you would use the comment feature of either blogger or Linkedin to share your insights on what is most important, and or mention what you feel is missing. In return, I will attempt to distill the information into a condensed format that will be available for the community to use.

Distillation as of 10/29/16:
A successful CISO must work constantly to build relationships. A simple key to this is being the type of person other would want to have a relationship with. The basics are fairly obvious, good hygiene, dress for success, smile, be a good teammate. But there is another aspect; be genuine. There is no point asking "how was your weekend" or "how are the kids" if you don't care.  A simple measure of caring is when was the last time you wrote a thank you note. And you don't have to care, you can be a fine engineer, coder, or network analyst if you don't care about you co-workers, but you can't be a successful CISO.

Mindtools defines the characteristics of good work relationships as

• Trust 
• Mutual Respect
• Mindfulness 
• Welcoming Diversity 
• Open Communication 

Joel Garfinkle's 10 points:

•     Share more of yourself at meetings. 
•     Speak positively about the people you work with, especially to your boss. 
•     Be supportive of other people’s work. 
•     Ask others to become involved in your projects or activities. 
•     Write thank you notes. 
•     Initiate conversations by asking questions. 
•     Initiate repeated interactions and communications. 
•     Participate in activities with others that don’t involve work. 
•     Share information. 
•     Introduce yourself at social work events. 

Inc, has a blogpost with 9 points:

• 1. Take the hit.
• 2. Step in without being asked.
• 3. Answer the question that is not asked.
• 4. Know when to dial it back.
• 5. Prove they think of others.
• 6. Realize when they have acted poorly.
• 7. Give consistently, receive occasionally.
• 8. Value the message by always valuing the messenger.
• 9. Start small... and are happy to stay small.

I put this out on Linkedin and here are some of the comments:

Dr. Shawn P. MurrayEthics first, build solid relationships with all business units and work with the board to ensure information security is part of your culture and corporate governance.

Bill CarrI think every employed person needs to build relationships so that is a given, you earn trust through communications, competence and character. CISO should above all else be Courageous; they must hold the line even when it could mean they need to find another organization to work for that is genuinely committed to protecting its data, its customers, its brand, its shareholders and most importantly its brand (good name). The business nor anyone else should be able to tell the CISO how to protect the organization for on his/her head the sword of Damocles will fall if it goes bad.

Bill CarrTraits: Courage, Character, Passion, Ethical, Disciplined, Focused, Decisive, Reliable, Resourceful, Assertive, Critical Thinker, Analytical, Effective, Influencer, Listener, Learner, Knowledgeable, Experienced (Wisdom), Smart, Visionary, Logical and Organized.


Stephen Northcutt is Director for Academic Advising at SANS.EDU and chair for SANS Rocky Mountain 2017.

Important Knowledge Skills Abilities (KSAs) for successful CISOs

For the past few months I ran an open survey of Linkedin connections and the GIAC Advisory Board to determine the Knowledge, Skills, Abilities, (KSA) a successful CISO must have. Then we ran a survey, (thank you Barbara Filkins), to measure the Importance, Criticality and Frequency of the skills. This post covers the importance, (core), KSAs.

The 61 survey participants ranked each element between 1 and 5 where: 1 = Least important 2, 3 = Important  4, 5 = Most important

Tier 1 Most important

• Able to build relationships and partner with Directors and Board members, particularly outside of IT 4.51
• Ability to understand the balance between risk and security, and how to integrate this into a given organization. Able to facilitate discussions about risk. 4.44
• Clear understanding of which battles to fight, ability to prioritize when there is always more work to be done than time or resources. 4.38
• Able to integrate with the company's mission, and with other division's agendas. Understands the business. 4.36
• Build and take care of the team you are privileged to lead 4.33
• Ability to effectively communicate technical information to non-technical audiences.   4.32
• Ability to tie business context/concept to data protection and technical components within the IT and InfoSec space. (Note: A person doesn't need to be an expert, he/she needs to be able to connect these dots for other business leaders). 4.30
• Liaison between technology and the business, collaboration focused. 4.30
• Ability to attract and retain top tier technical talent as well as develop less experienced team members 4.23
• Work well under pressure 4.18
• Broad knowledge of cybersecurity principles, ranging from technical to human to physical. 4.10
• Creative thinking, able to apply adaptive strategic and tactical thinking. 4.08
• Mentor and communicate (up and down) 4.02

Tier 2 Important

• Capabilities focused, not vendor/tool focused 3.98
• Life long learner 3.97
• Humility, capability to admits that he/she doesn't know everything 3.95
• Build and manage relationships both inside and outside the organization.  (Think Tipping Point connector.) 3.85
• Incredible organizational ability to keep people on task and focused in order to build, design, deliver, and expand the information security program. 3.79
• Ability to grow the team as the organization grows. 3.79
• Ability to prioritize and triage incident response and vulnerability remediation in a calm balanced manner. 3.74
• Deep knowledge of the cyber threats we face today and tomorrow. 3.60
• Ability to produce and manage to a budget 3.59
• Design, build, manage an effective security awareness 3.58
• A sense of humor. 3.39
• Expert experience in at least one cybersecurity discipline 3.13

Tier 3 Less Important

• Resume demonstrates loyalty to organization, a good CISO does not job hop 2.74
• Advanced degree (Note: Masters degree preferred, especially MBA) 2.08

About the survey

Job roles of survey participants

Industry of survey participants

Organization size of survey participants

Geographic regions of survey participants

What is next for this project?

I am going to try to use Linkedin and the GIAC Advisory Board to collect some qualitative information to see if these KSAs can be broken down further and also to seek wisdom on how to approach the most important KSAs.

About the author:

Stephen Northcutt is Director for Academic Advising at the SANS Technology Institute and chairperson for SANS Rocky Mountain 2017, June 12, in Denver.

Qualities of an effective CISO (1)

‪SANS.EDU‪ is updating our strategic plan. I asked my Linkedin network for help in documenting what knowledge, skills and abilities an effective CISO would need to have. The list below is my best effort to create a synthesis of the replies. Please note, the rank order does not imply importance, I tried to group similar things together, but it will be a future step to rank these in importance. Thank you very much to everyone that participated.

1. Broad knowledge of cybersecurity principles, ranging from technical to human to physical.
2. Deep knowledge of the cyber threats we face today, and tomorrow.  A CISO must understand the motivation behind malicious actors, and knowledge of the techniques, tactics and procedures used can help us better defend, detect, isolate, and recover from the inevitable.
3. Ability to understand the balance between risk and security, and how to integrate this into a given organization.
4. Expert experience in at least one cybersecurity discipline
5. Incredible organizational ability, keep people on task and focused in order to build, design, deliver, and expand the information security program.
6. Ability to prioritize and triage incident response and vulnerability remediation in a calm balanced manner.
7. Ability to effectively communicate technical information to non-technical audiences.
8. The ability to tie business context/concept to data protection and technical components within the IT and InfoSec space. Doesn't mean they need to be an expert; means that they need to be able to connect these dots for other business leaders.
9. Liaison between technology and the business, collaboration focused.
10. Clear understanding of which battles to fight
11. Capability focused, not vendor/tool focused
12. Strong leadership skills at both the organizational and individual contributor levels.
13. Relationships (inside the org and out), (think Tipping Point connector)
14. Ability to attract and retain top tier technical talent
15. Mentoring (up and down)
16. Taking care of the team you are privileged to lead
17. Creative thinking, able to apply adaptive strategic and tactical thinking.
18. Life long learner
19. Humility
20. A sense of humor.

The "oral history" of GIAC

A couple days ago press@sans received a note asking for some information. Apparently, a professor is writing a book about the origins of cybersecurity and wanted an oral history. Apparently, some of them get added to the Charles Babbage Institute, here is an example with Lance Hoffman. So, I took a crack at it as shown below.

In 1981 I graduated from Mary Washington with a BA in Geography, (cartography and air photo interpretation), and was hired by Defense Mapping Agency, DMATC, (now NGA). They were just converting from manual processes to Computer Aided Design, (CAD). The terminals were powered by DEC PDP 11/70s. I was working the evening shift, essentially 4 - 11 PM. The facility was on the Potomac river gorge, (Brookmont, (Bethesda), Maryland), and in June, we had a thunderstorm almost every day and we were required to shut the systems down and unplug them; they even had a thunderstorm code to charge our time to. There was no IT department for technical support in the evenings, so during the thunderstorms I started reading the manuals and even started coding in RSX-11M. The path to being a technical GS-12 involved getting a masters degree in geodetic engineering from Virginia Tech. So I started taking courses at Telestar Court in Falls Church. One of the required courses was data structures in Fortran, my first computer course with punch cards on an IBM 360 mainframe. I loved it. I quickly took all the computer courses. When I signed up for one as my elective, the topography department head denied it. He said, "You have taken enough computer courses". I updated my resume, (forget what they called the form back then), and got an appointment with the computer department head. He read my paperwork, looked up and said, "Do you have any experience with small computers?" "Well sir, I am the president of the Fredericksburg Commodore 64 club", I replied. I was hired, I became a computer guy and have never looked back; who would want to do anything else?

In 1997, my primary security focus was network intrusion detection. Fred Kerby, cybersecurity manager at NSWC Dahlgren, had arranged for me to speak at a conference, but the event was canceled. He called Alan and asked if I could give a talk at SANS Network Security in New Orleans.

Michele Crabb, (now Michele Guell), assigned me a topic, "Making the most of your opportunities with management". I was stumped. I was terrible at working with management. So, I chose the angle of talking about all the mistakes that I had made. I didn't think anyone would come to the talk, it was the last time slot on the last day; hundreds of people showed up and it turned out to be the highest performing short talk at a SANS event.

After the scores came in, Alan Paller found me and asked me if I could speak on a technical topic, so we talked about intrusion detection that night. They put flyers all over the hotel and brought in a wooden canoe with local beers on ice. The Q&A went in the direction of searching a database of network information looking for patterns. A couple of years later, the notes from that discussion led John Green to create Dark Shadow. It was interesting to see how many organizations were toying with the concept that would later be known as SIEM.

In the center of the room was Marcus Ranum, with his famous red cowboy boots. I was a bit intimidated.  Marcus didn't slaughter me and later, when Marcus was with NFR I helped Kent Landfield extend the NFR N language to support intrusion detection patterns. Now I was writing for two different IDSes and was beginning to realize that if you knew cybersecurity from an architecture perspective, you could apply it to multiple implementations.

In 1999, I agreed to write a book on cyber security intrusion detection, (link is 3rd edition, couldn't find first). I worked on it on the commuter train up to the Pentagon from Fredericksburg VA and then on the way home. After supper, I would retire into my office and write till about 10 PM. Every couple of weeks, I received a call from Alan Paller. He wanted to do something to “prove people could do the job” in cybersecurity. I was pounding on the keyboard and he would tell me he hired, this famous person and they could do it, then a few weeks later, he would call and say it did not work out. I liked Alan and would have loved to help him, but that wasn’t my mission, the book was my mission . . . until two things happened.

First, even though I had moved to the Pentagon for missile defense, I was still working with the Navy Laboratory at Dahlgren Virginia. I used some of my training dollars to send two of the Shadow IDS team members to the USENIX 99 conference in Monterey. I was there as well. The talk I had chosen was boring, so I drifted into some of the other talks. I did not see my people. I wandered through the Portola, (then Doubletree) hotel and conference center, did not see them. Finally I ended up on a deck overlooking the bay and saw them. They were in sea kayaks. That hurt; I had limited training resources. And it hit me. Mr. Paller’s idea could also tell an employer that spent training money on someone whether they actually mastered the subject matter. I still was not totally invested, but was becoming interested in the idea of proving someone could “do the job”.
NOTE: this conference was also what got me thinking about the flawed structure of "technical conferences", 1 and 2 hour talks leading to more depth at ShadowCon and later the track system at SANS.

Second, back at work at the Pentagon, I found out my IDS contractor had resigned to be part of a startup with the Enterasys Dragon IDS. Our prime contractor handed me a stack of resumes. One looked really good, lots of experience, remarkable, because in 1999 intrusion detection was in its infancy. So, I told the prime to bring him on board. His first day on the job, I wanted to bring up a RealSecure IDS on one of our new facilities. So, I handed him the disk and told him to load the maximum signature set. I came back a few hours later and it was not running. All you had to do was load the disk, agree with the Microsoft install wizard, next, next, next, choose the signature set and you should be up and running. Long story short, his resume was bogus, I don’t even know how he managed to write it. Mr. Paller’s vision was starting to make a lot of sense to me at this point; I called Alan and told him I was in.

Alan, came down to visit the Shadow team at Dahlgren and we spent some time on the whiteboard. Security was getting more complex, even in 1999 there wasn’t such a thing as a “security guy”. Instead, there was a firewall/perimeter expert, IDS expert, Windows OS, Unix OS, forensicator and so forth. To prove someone could “do the job”, we would have to define the job. Then break it down into knowledge elements, knowledge, skills and abilities, (KSAs). 

NOTE: as GIAC came to be, KSAs morphed into ICF, (important, critical and frequent) values.

As 1999 drew to a close, the white house security council requested my participation in Y2K in the event cyber attacks were going to happen. My boss at the Pentagon didn’t like it, but couldn’t really say no, so I reported to the Gerald Ford House Office Building to set up shop. It was a terrible experience. The FBI wanted to be in charge, the US CERT wanted to be in charge, the GSA point of contact was really mean. I was just a techie, unprepared for the worst of government politics and turf battles. Mr. Paller was kind enough to take over on site and I did all the work remotely including setting up multiple global response centers counting on Richard Bejtlich, Arrigo Triulzi and other analysts. I had been a happy government employee till that event, but I was done and SANS was kind enough to hire me. I resigned from missile defense January 5, 2000.

For the next year, I focused on gathering knowledge about the security skills and figuring out how to teach and test it. Eventually we settled on the name and idea of GIAC, (we had created the brand earlier as the Global Incident Analysis Center, later, incidents.org, now known as the Internet Storm Center), and rebranded as the Global Information Assurance Certification. The early days were rather crude, essentially two guys and a dog writing test questions, but we focused on continuous process improvement. 

We all know the events of 9/11/2001, but what most people don’t know is that it spilled over into cybersecurity, probably in part due to the Code Red worm two months earlier. All of a sudden, we were facing enormous demand for our training and attempts at certification; every class was sold out. Capacity was maxed out. A SANS employee, Zoe Dias, spent weeks figuring out how to increase capacity by a factor of 10, she would wake up in middle of the night as ideas came to her and she continued to chip away at the logistics problems. Her work resulted in the distribution system we depend on today.

In 2002, the industry was realizing there was a lack of provable security skills. Steve Katz, CISO CitiCorp, had done a briefing on the topic that got picked up by the press. Eventually, GIAC was lucky enough to hire an engineer named Jeff Frisk and he cared enough and was detail oriented enough to help GIAC become what it is today.

Search Engine Optimization and Security Certifications

10/24/16 I typed "security certification" into Google using the Firefox browser. Observations:
363M results
6 paid ads: ISACA, CTC.EDU, ASIS, Denimgroup, Cybervista, apus.edu (I had never heard of most of these before)
Top organic hit is Comptia Security +.  NOTE: Security + is mentioned 4 times on page 1 Google
Top article: "Darkreading's 10 Security certifications to boost your career"  They list the GSEC first.
0 mention of GIAC on page 1 of Google results

Initial recommendations:
1) Do not pay for ads, it is a crowded field so it will cost money and yield minimal results
2) Create a short list of distinctives, suggested examples:

• GSE, most advanced cybersecurity certification, this is halfway down the page for search term GSE.
• GIAC Advisory Board, best home field advantage in the industry, (my blogpost was 2nd hit)
• Cybersecurity skill specific certification, that Google search term does not mention GIAC and ISACA is second organic hit, "ISACA is First to Combine Skills-based Cybersecurity Training with Performance-based Exams and Certifications to Address Global Cyber Talent Shortage"
• ETC, ETC

2A) Build content for your desired distinctives linking back to GIAC. I am modeling that behavior in this blogpost, (note that I am not linking to "the other guys).

3) For each GIAC certification, type the search term into Google. For instance for the search term "GPEN" the first hit is a vaporizer. Build an article about the certification that links to the appropriate cert page.

What is a cybersecurity architect, (and how to hire one)

According to the Burning Glass report titled Job Market Intelligence: Cybersecurity Jobs, 2015, 5% of all cybersecurity job postings are for a job title of Security Architect. Unfortunately, the industry is still unclear as to exactly what an IT Security Architect is. According to Payscale, the median compensation is $114,000/year, (which sounds a bit low). The concept is, however, starting to mature. Certifications are being developed for IT Security Architects, and training courses are offered by various organizations to help prepare one to be a Security Architect. The (ISC)² organization has created an ISSAP (Information Systems Security Architecture Professional) certification. The SABSA organization offers a set of integrated frameworks, models, methods, and processes, used independently or as an integrated enterprise solution.  Monster.com listed a job description for a Senior Security Architect, that lists the following skills:"Network Security, Network Hardware Configuration, Network Protocols, Networking Standards, Supervision, Conceptual Skills, Decision Making, Informing Others, Functional and Technical Skills, Dependability, Information Security Policies".

 The TOGAF, (open architecture), certification has to do with thinking like an architect. A security architect needs to be able to function as a general systems architect for the enterprise. Without the big picture, it’s hard to provide big security solutions.

A security architect should have the ability to conduct "as is" process gap analysis, (where are we now, where do we need to be, how to get there). They generate technical implementation and management prioritized guidance that includes evaluation tests and metrics such as those identified in the CCS/CIS Critical Security Controls. The implementation is a cooperative effort between business management and the Security Architect who brings needed experience, expertise, and consultation to the decision-making process.

Engineer and Architect compared

Architects know what needs to be done to get you to end goal, engineers know how to do details of next tactical step in the project.

Architects tend to think in concepts; defense-in-depth, least privilege, breaking the exploit kill chain.  Engineers tend to think in products; Firewalls, IPS, Anti-malware, file integrity monitoring, DLP, etc.

Architects worry about how the ecosystem works together, engineers worry about how to keep things running and working.

An engineer can tell you how to design your network. An architect can tell you why it should be designed that way, and will be able to suggest changes based on your specific needs.

An engineer can tell you which protocols companies should use for discrete tasks. An architect can explain why those protocols make the most sense, and can usually detail the previous state of the art.

Architects want to know exploit vectors and what intellectual property was exfiltrated from the company, engineers want to collect evidence and remediate.

Architects think vulnerability management, engineers think patching, hardening and scanning.

Architects think big picture and are good presenters and salesmen of security ideas to upper management, engineers are where the rubber meets the road, (the real problem solvers in the trenches).

You have to have both, most security professionals function better on one side or the other, nothing is worse than having an architect that only wants to engineer or an engineer who only wants to architect.  However, many companies struggle because they ask a single person to do both and then are frustrated that that person has a weak spot on one side or the other. 

The key attributes of an architect in order of importance:

• Analyze the business operations of the organization and map them to data flows between the information processing zones within, as well as to and from, the organization.
• Design a security solution which suits the risk appetite and the real threats the enterprise faces. They use the basic classes of cybersecurity tools available, such as perimeter protection, detection, OS protection, identity management and SIEM style information correlation to implement defense in depth at choke, or control points, of the enterprise.
• Understand the "big picture" in terms of all IT systems, if you don't securing them is impossible. In critical infrastructure organizations, understanding the physical security controls is crucial and architects may be "dual hatted", cybersecurity/physical security.

Ideal persons to help interview a candidate for an architect position include: the IT manager, (such as CIO), security manager, (or CISO), network manager, (or senior network analyst), systems manager, (or senior systems administrator), applications manager, (or senior software developer with a cybersecurity interest).

Interviewing an engineer for a network architecture position

We interviewed a number of GIAC Advisory Board members who have been working as architects for major enterprises as to what they look for an architecture position. They recommend that you be careful about giving candidates a real world problem (even pretending it is 'made up') as this could be dangerous to a company either from a PR or security perspective if it got posted on the Internet in some way. There are a number of practical assignments defining a mythical company called "GIAC Enterprises". If you Google that term you can get some scenarios to use for the exercise. Here are some questions they recommend asking:

Do you have a home network setup? Please describe it to me.

When designing an architect/infrastructure for security we have to be at least "aware" of the various protocols/technologies used within Corporate America. Please tell me a bit about:

• Equal cost paths for egress traffic
• High Availability Design issues
• Packet shaping
• The role of the network in compliance
• What ideas do you have to improve our DR/BCP

Please tell me a bit about each of the technologies below and when and why you might use them:

• OSPF
• EIGRP
• MPLS
• RIP
• GRE
• IPv6
• Proxy ARP
• Static routing

Give them just the hex of an IPv4 packet or a DHCPv6 trace and ask them to tell you what is going on. They don't have to be packet ninjas, but they should know what is going on. We teach managers to do this with prospective employees, (in the course we author and teach, Management 512).

Interviewing an engineer for a security architecture position

• What threats do you perceive in this company's environment?
• What are the assets and/or business processes (5 maximum) you think are the most critical ones for the organization?
• What assets do you think are the most exposed?
• Identify the weakest links in the system as a whole (Networks, Systems, Applications, Data, Users).What basic access controls would you design into the network (relevant to my business)
• What if any IH procedures would you put into place regarding the network.
• If you were an attacker, what would you be after?
• If you were an attacker what would your business model be? That is, how can an attacker make money by attacking us.
• If you were an attacker, how would you go about penetrating us?
• What architectural solutions (Protection, Detection and Reaction) would you propose for the different components (Networks, Systems, Applications, Data, Users) to address the threats and mitigate the risks?
• Draw for me a high level (network) diagram that shows your proposed architectural changes and solutions.
• Develop an implementation plan for those solutions (short/middle/long) term.
• Out of the solutions you mentioned, what are the 5 ones that add the greatest value?
• Show me how would you adapt your solutions and what would you prioritize according to different budgets: $1.000, $10.000 or $100.000
• What are the solutions that you think would be more difficult to implement (due to technical, budget or cultural reasons)?
• What policy / cultural changes do you think are needed (if any) for your long-term plan to succeed?
• Propose a couple of security solutions that would enable this company to improve business by doing something it can't currently do.

More general questions

• If we are looking more of general purpose architect, consider some of these questions. If our organization wants to field a new ecommerce sites, can you describe a couple different scenarios or approaches to the architecture. What are the primary tradeoffs between architectures? What vendors would you use and why?
• Get your technical folks to help you identify a real world problem your organization is facing. Can the candidate engineer a "duct-tape" solution to temporarily address the issue. You don't want a candidate that is always relying on spending $$$ to accomplish a task.
• Please explain a recently announced vulnerability of your choice, and what solutions you would implement to mitigate the threat.
• Here is a whiteboard and some markers... draw me a diagram, design, or something of your choice using these tools to communicate a concept, architecture, or something of your choice.
• Tell me about your experience with the open-source movement. What sources do you use to find information on new products related to network monitoring?
• If there was a network problem, what are the basic steps you would go through to in order to troubleshoot the problem?
• What architectures, software, or deployment strategies have you used successfully in the past, but would no longer use? Please tell us why.
• Sell us on yourself. What are your strongest personal assets? What specific attributes would you be bringing into the organization that will make a positive contribution to our overall success?
• Tell us about an instance when you had to communicate an idea/process/procedure to a customer that you know will be resistant to you. What was your initial approach? Did you have to change your approach? What was the outcome?
• What approach do you take when you need to learn about a technology? Do you consider yourself a life-long learner? Why?
• What was the one question we did not ask that you came prepared to answer?

Sample Candidate Profile & Requirements
Candidate has substantial experience researching, authoring, and implementing security configuration standards across multiple platforms. Candidate's experience includes a successful track record of evangelizing standards, managing and/or creating the standards compliance and remediation processes, as well as presenting the value propositions of standards-based security management to senior managers within a Fortune 500 organization, or similar scale environment.

The self-directed individual represents COMPANY as a participant in industry working groups and standards bodies. Candidate's familiarity with security industry standards, working group processes, and content lifecycle management adds great value. Active participation in - or contribution to - OASIS, Liberty Alliance Project, NIST, Center for Internet Security, or other similar open forum working groups and committees demonstrates candidate's ability to advance COMPANY's concerns within the broader security industry.

Candidate is familiar with threats, vulnerabilities, and exposures across diverse systems, and successfully communicates this data in terms of operational risk and business relevance. Candidate brings to COMPANY extensive background creating and executing closed-loop vulnerability management practices, and can leverage such experience in coordinating individuals with competing priorities across multiple departments to mitigate risk.

The ideal candidate has 5-7 years experience in the industry. Familiarity with types of products offered by COMPANY, and the core business processes needed to deliver services, is essential in making security relevant to the lines of business the team supports.

Candidate can demonstrate a proven track record of communicating and working proactively and professionally with internal and external auditors, and other groups responsible for ensuring that an organization is properly protecting the interests of its customers, shareholders, and employees.

Candidate is familiar with software development lifecycle methodologies. Demonstrated experience gathering and documenting business and technical requirements for implementation by internal development teams and/or external vendors shows that candidate can lead others in meeting COMPANY's security requirements.

Candidate must bring extensive experience leading and/or significantly contributing to cross-departmental technology projects. The candidate leverages an understanding of industry-standard project management methodologies, experience with project financial controls, and the ability to communicate the financial justification for security projects to deliver on COMPANY's Information Security Strategy.

Candidate has led, or significantly contributed to, enterprise projects to deliver security information management solutions. Candidate shows experience building an infrastructure to aggregate, deduplicate, and correlate massive streams of security log data; candidate has delivered processes and procedures to triage, analyze, and take action on such information; and candidate has designed management reporting to instrument and continuously improve security information management.

Candidate's significant experience with network security controls such as routers, switches, firewalls, intrusion management solutions, network access control, and related solutions is required when coordinating delivery of holistic security in partnership with COMPANY's Network Engineering group(s). Extensive understanding of network protocols, data flow analysis, and network design and troubleshooting assist the candidate in leading others to successfully deliver a security program.

Candidate's familiarity with application security practices such as secure coding and secure development lifecycle management is required in coordinating with application architecture and development groups, as well as positioning system security in the broader context of COMPANY's information security program.

Skills and background in computer programming are desirable, but not required; however, candidate must demonstrate knowledge of design patterns used in enterprise applications. Understanding of how applications are developed, deployed, and managed is essential to demonstrating that candidate can design security solutions to protect critical assets and data. Familiarity with security principles in Service Oriented Architecture, WS-Security standards, application frameworks (.NET Framework & J2EE/Java EE), and the use of cryptography in applications ensures that the candidate can explain complex issues.

Certification by industry standard certification bodies is encouraged, but not required. SANS/GIAC, ISSAP, or similar certifications will be considered as evidence of candidate's dedication and commitment to demonstrating an objective baseline of skills. However, keep in mind that according to Burning Glass 35% of all security job postings require a certification.

Candidate has 3-5 years experience designing, implementing, and measuring closed-loop security management workflow systems. Proven experience integrating security controls into enterprise workflow and incident/problem management systems is paramount in successfully delivering on the goals assigned to this position.

Acknowledgements

J Michael Butler who was a great help in previous versions
Roland Grefer helped me clean up the writing
Chad Lorenc really beefed up the architect engineer comparison
Thomas Williams TOGAF and the importance of physical security

References: All links valid 10/5/16 unless otherwise noted

http://burning-glass.com/research/cybersecurity/
http://www.payscale.com/research/US/Job=Security_Architect,_IT/Salary
ISSAP®: Information Systems Security Architecture Professional https://www.isc2.org/issap.aspx
SABSA (Sherwood Applied Business Security Architecture) www.sabsa-institute.org/
http://www.sabsa.org/node/73
http://hiring.monster.com/hr/hr-best-practices/recruiting-hiring-advice/job-descriptions/senior-security-architect-job-description.aspx
https://www.cisecurity.org/critical-controls.cfm
Information Security Forum (ISF) https://www.isfsecuritystandard.com/SOGP07/index.htm ***No longer worked when checked December 3, 2012
Department of Defense Architecture Framework (DoDAF). retrieved 10/5/16 (thanks to Chris Holabird)
Department of Defense Architecture Framework (DoDAF) v2 2009
http://jitc.fhu.disa.mil/jitc_dri/pdfs/dodaf_v2v1.pdf
http://jitc.fhu.disa.mil/jitc_dri/pdfs/dodaf_v2v2.pdf
http://jitc.fhu.disa.mil/jitc_dri/pdfs/dodaf_v2v3.pdf

Department of Defense Architecture Framework (DoDAF) v2.02 2015
http://dodcio.defense.gov/Portals/0/Documents/DODAF2/DoDAF%20v2.02%20Chg%201%20Vol%20I%20Final%202015-01-19.pdf
http://dodcio.defense.gov/Portals/0/Documents/DODAF2/DoDAF%20v2.02%20Chg%201%20Vol%20II%20Final%202015-01-19.pdf
http://dodcio.defense.gov/Portals/0/Documents/DODAF2/DoDAF%20v2.02%20Chg%201%20Vol%20III%20Final%202015-01-19.pdf
http://dodcio.defense.gov/Portals/0/Documents/DODAF2/DoDAF%20v2.02%20Chg%201%20Vol%20IV%20Final%202015-01-19.pdf
Zachman Institute for Framework Advancement (ZIFA) http://www.zifa.com/framework.html
NIST - Managing Risk from Information Systems http://csrc.nist.gov/publications/nistpubs/800-39/SP800-39-final.pdf