Finding the balance: Productivity and innovation in IT’s AI era

our very own CPO at a tera I am gonna um head it over to you tell us a bit about

um who you are um what a tera and whz um are exactly for anyone that doesn’t know

but I doubt it uh Ryan go ahead sure um hi everybody

thank you so much for the chance to uh join you all virtually today again my name is Ryan gazans and I am the CIO and

ceso at whiz uh if you’re not familiar with us whiz is a platform that we buil to help team securely develop build and

run uh everything they put produce in the cloud um be it on AWS Azure gcp um

and our focus is really on democratizing security across all of the teams that have to partner to efficiently and

effectively use the cloud across R&D devops and security um as C and ciso U

my roles include leading our it function um which includes it engineering Ops and business applications uh as well as our

security teams which are responsible for security engineering operations uh risk and compliance um I’ve been at whiz for

about two and a half years now uh and previously came from uh Facebook now meta um and before that tum and

mandia over to you t uh thanks um I’m T I’m the chief

product officer in uh in atera um atera is an AI first remote it all-in-one

solution uh we’ll probably touch on this uh later on during uh during the webinar

many of you already familiar with with AA um in my uh previous roles I’ve been

the VP of product at redis uh and also served uh several uh senior roles uh in

product departments uh converts and Flash networks couple of others um fun

fact I started my way as a data scientist around computer vision so I

have uh some U some very warm Corner in my heart for uh for AI and data science

so thank you thank you both and maybe quickly I’m Anna by the way I missed

that part I’ll be moderating today’s session I lead the uh the content and

Communications team here at at Tera and I’m very excited um to kick this off um

just a quick minute before we start I want to go over a few housekeeping rules um this webinar is being recorded um of

course the the recordings of the session are going to be shared with everyone that registered for the webinar um you

can drop your questions or comments as mentioned for Ryan and tal in the Q&A

section please please help us uh stay organized um so we can go by order we’re

going to have 15 minutes at the end of this session to take all your questions

um that’s it let’s let’s uh Dive Right In so um like I said I’m super excited

to have you both here um and as we’re going to explore how AI serves as a

powerful Catalyst for enhanced efficiency and streamlined operations um

how it enable to focus on more strategic initiatives um I also want to maybe

touch on um the complexity and the challenges that organizations today must

navigate carefully when introducing um this new layer of AI um as we know this

can lead to um integration issues need for upskilling it might change uh the

Team Dynamics so we really wanted to take today um to kind of discuss how AI

um drives the significant prod productivity gain while also addressing some of these complexities so um I’ll

I’ll start um so as this technology continues to evolve how would you each

describe its transformative role um in enhancing productivity across Industries

and specifically within it departments today cool um I’ll start um so so um so

maybe a little bit of forward um you know it’s been a lot of big words around

you know uh productivity and complexity and and a lot of things and and I think uh from my perspective the main the main

issue here is to learn a little bit about you know about Ai and what’s it

going to what’s the impact is going to be on your organizations if there is an impact I think uh uh it’s a it’s a big

if um and and when looking at the at um at what AI has done already um I would

probably look at it from uh you know from the micro level I would look at uh

at the impact on GDP and I think that to date it has been nothing and um there’s

no you don’t see a big impact on GDP on the micro level so it’s very very hard

to see uh a real impact you know there’s a lot of hype a lot of big words on this

but there was no real impact on on GDP so I I don’t assume that there’s a lot of impact on uh on productivity around

companies I do however think that it will it’s going to uh have an impact uh

the real question is how and when uh and there’s a there’s a there’s a saying by

uh by KES who was a famous economist that says that said uh uh you know it’s

easy to know what’s going to happen finally at the end everyone dies the real question is when right so uh I

think I think it’s uh it’s a similar question here yeah i’ I’d agree with with T I

think look like if you look at the demand curve um even just on the consumer side let’s let’s set aside

business all of the major services like chat GPT are are basically still doubling doubling their user base

year-over-year um so the demand growth is there um there’s a lot of

experimentation I’d say especially when it comes to Translating that from casual usage into something that is deeply

integrated and part of how you as a business are being productive and I think the disruptive effects are

materializing more immediately in some Industries and areas than others so like clearly in areas like creative content

or areas where we’re improving existing agent-based systems like support chat vots where there’s a higher tolerance

for some error and hallucination like it is having I’d say like a a a small

impact there and growing quickly and that’s also the areas I think where a lot of people have questions about like

well what does that mean for jobs and economic growth um for expert systems in use cases I think it’s still on the

experimentation stage and where Enterprises are excited but being cautious because the reality is well

like these these features and the promises appealing they’re coming at a cost uh we’ll talk about this but adding

vendors incurs complexity in debt um none of the products that are integrating or

providing these features are doing so for free um and of course because they’re in part passing along the

charges from some of the major AI platforms that they themselves are using um a lot of organizations have to start

thinking about like the cost benefit analysis and can’t just blanket adopt all of the AI features for all of the

SAS vendors they use right out of the gate um and so that type of cost benefit analysis and rooting it back in

requirements um is what you’re going to see a lot of in the coming year uh again no company is going to be able to just

pay for every SAS vendor they use that provides AI enabled features to just turn it all on and use it across the

board and that’s good like it’s going to lead to more judicious usage and um we we can certainly talk more about how

that cost benefit analysis will play out yes we’ll get to that a bit later

maybe um yeah maybe starting from uh you know the beginning um and T touched on

it a bit um saying that AI was everywhere um obviously it’s not going

anywhere but looking at this kind of past Year’s wave of AI products and

services um and from your personal experience obviously either working with AI vendors or or introducing um this

technology into um your own products um would you consider gen AI to still be a

hype or no yeah I mean I I’ll talk from our experience um person adopting it in our

own product so there’s a few areas within the whiz product we’ve experimented with and then ultimately shipped AI enabled features one is

around search so like if you have a cloud environment and we’re inventorying it and the risks in it it’s really nice

to have a simple way to ask what under the hood is a very complex graph query so a question like show me virtual

machines in my environment containing sensitive data that is exposed to the internet translating a a simple on the

surface question like that in natural egg would to the underlying precise system understandable query like machine

understandable graph query um is actually a very good use case for llms and one where you also can have some

tolerance for imprecision uh another example is like AI suggested remediation where um given a risk or a problem and

this is true in a lot of security domains you can ask the system to generate a recommendation on what to do

and so in whiz we have like uh issues that we identify as we process the environment to look for risks and then a

remediation engine that says look based on what we know about your environment and these findings and based on the

expert data that we’ve populated the system with here are the steps you can take to fix it and here’s some example

commands you could run or infrastructures code that you commit to do that and when you when you control

the experience and box it in like that um in a very tailored path for the user

you can get a lot of value um so I think that’s where we’re going from experim M

mentation to finding value I think the flip side of that is a lot of the like hype around fully autonomous agents that

can just autonomously solve any class of problem and reason um at a very high

order we’re we’re still a few Generations away away from that and I think that’s where people who are not um

practitioners using these Technologies dayto day sometimes buy into some of the hype from all the investment going into

the space and realize that like in practice a lot of these systems are still a bit goofy um still hallucinate a

lot still make mistakes that are just not acceptable in certain production usage and that’s where I think a lot of

orgs will have reality checks and shape their Investments um while still keeping close tabs on how these products are

evolving um yeah I I agree with uh with Ryan I’d like to touch on this maybe from a from a different uh a different

uh angle maybe um so when you look at uh at at different a Solutions out there

and you probably you know I see every day like uh tens of uh of commercials

for different AI solutions for for every vertical and every specific use case um

I think you look at two different use cases uh the first use case is uh is

what I call enhancement so you have people working and you want to enhance their work this is a this is a use case

that’s hard to quantify and many times the business model around this is a

business model that uh that’s uh that’s not onetoone with the benefit it’s it’s

very very hard to quantify it by the way um in a survey that was done by by

McKenzie I think the number one concern for cios was how to quantify the uh the

value of AI Solutions um and there’s another type of solutions and and that type is is is an

actual save usually of human labor it it doesn’t have to be human labor it could

be other other types of uh of savings as well for example replacing a large

system that costs a lot of a lot of money with a with the smaller system that costs less but but essentially you

see the ROI very very quickly so and and and many time this is also associated

with the business model so if you have a business model uh for AI that’s uh

that’s uh uh not uh software as a service but Service as a software meaning that you actually charge based

on success then this is something that allows you to to uh to adopt an AI

system without bearing the cost of risk which is usually what we uh what we uh

what we look at when when you look at all those all of those commercials so so

I would say that you know if you look at those AI Solutions I think the first ones to succeed are the ones that show

clear Roi it’s very very hard uh but again if it’s success-based it’s it’s

much easier to show Roi so yeah I would say this is this is where I would go

first what what’s interesting about that T I think is the the companies and orgs that have the broadest data that could

help with that ROI analysis are also the companies that are selling AI enabled products and services and there or have

somewhat of a vested interest in showing how amazing it’s being um and like not

picking on any company in particular here but like I’ll take an example of why it’s so hard to get that that Roi

data um a week or two ago headlines were made during Google’s earnings call when SAR announced that 25% of code at Google

is now being generated by Ai and what when I heard that percentage from from me it was like well that that doesn’t

tell me much because one like 25% is measured by what like lines of code for net new commits like what’s the the the

denominator two if if you’ve ever worked in as a developer in a large code base you know that a lot of code is autogen

either by template or by things that are like various forms of um like enabled

autocomplete type functions and so just saying like AI is now generating 25%

code doesn’t actually capture a delta in a way that like on the surface it might seem to of how much things have changed

or how much they’ve improved and so um I think you as you said it’s going to be a while before the consumer side like the

big organizations trying to use this actually have the the data at the back where it hasn’t has not made a

meaningful impact on efficiency and productivity I I think we’re ways from that yeah this this is so true and I

think that you know if if if you look at a success-based uh business model which

is which is starting to loom um you know this is this is kind of you don’t need

to prove the ROI so you only pay if uh if there was a success so so if you look

at systems for example that uh that solve uh uh tickets for for users

there’s uh a tera is going to launch this in a couple in a couple of months but let’s say that you have this kind of

system if you pay me only for Autos solved ticket a ticket that was solved

with no uh intervention from a technician that means that you have nothing to lose I don’t need to prove

the RO just Implement that and just pay for whatever he took and and and you know what’s the value is what the value

is for you and and if if the if it costs less than the value it’s it’s an no-brainer I would say that so I would

say that that that you know systems and business models that start this way

would probably be the first to succeed because it it eliminates the barrier to entry for for cios that are worried

about the uh the ROI I agreed it’s it’s a lower risk way to sort of dip your toes into it and and see the effects yep

yeah yeah and definitely uh okay maybe that’s a nice segue you

tell me um so will companies and teams that adopt AI earlier would they see

those gains coming in earlier are are they going to be able to to prove um Roi

before others or um would that still you know remain out there and what are the

gains of any early AI adopters yeah tough tough question

um so so um I think that past experience shows that it’s not always that the

first company to jump on the bwag on the bandwagon or the one that will succeed in the end um you know we’ve seen it in

in in in multiple examples and Ryan you and I talked about it earlier you know

there was there’s the example of the theom boom uh you had lots of companies

who remembers pet or I don’t know like various companies that everyone thought that is going to

be the next thing and and uh I don’t know if anyone thought that a book

seller is going to be one of the biggest companies uh in the world today so it’s

it’s very very hard to know who the winners are going to be uh and also

since then there’s a lot of companies that that came later on uh such as Facebook Ora uh uh that started later

after the doom boom and and they succeeded very well so it’s very very hard to know um I I would argue that um

you know if you look at uh at operations um there’s that kind of a

like a dervinis approach to it so if you manage to lower your costs uh the

operational costs then you have an edge over your competition in whatever field

you are and and if you have that edge this is this is an edge that accumulates over time so if if you start with with

AI and you do get which is which is arguably uh uh true and you do get this

uh performance boost that means that you have this this Edge over your competition and that means that you’ll

be able to uh drive better prices and then you you can capture the market better uh be more efficient uh and this

ultimately will allow you to uh to win your Market um I would say that it’s a risk but I’m

not sure that you can allow yourself not to take that risk uh just it doesn’t

work I I I think the answer has to come with the context of where you are as a

business right like if you’re if you’re a early early stage startup at you know

one or two person operation where everyone has to wear many many hats and you’re basically going from zero to one

in every function um I think early adoption of AI Technologies AI

supportive Technologies like is such a good for Force multiplier and at a point

where you’re you’re good with like draft stage prototyping stage capabilities and functions in every regard that like the

first mover Advantage is very real in that context because it’s it is going to help you get something meaningful to

Market quicker and like cover roles that you can’t have the luxury of dedicated resources or people uh or tools for so

in that context I think absolutely I think for established organizations it does tend to be more nuanced and this

goes back I think to just first principles around buying vendor driven Solutions in in any context like

organizations often struggle with defining technical requirements for

berse build uh uh situations and in an it org working with business partners

with different teams of use cases that are not technology experts it’s often very hard to Resource that well and so

when you take AI Technologies it’s the same thing where teams that are seeing their their SAS vendors offering AI

enabled product features or new startups that lead with that like are appealed by the the product message and and the use

cases but um I think an important role of it leaders is is to Center them back on like all right well let’s let’s think

about your requirements the problems you’re trying to solve the technology

that you have today and where it’s falling short or covering that and and think about the problem from

requirements and not from features um and that always like steers you in a good direction and helps I think

overtime orgs have some governance that avoids like accumulating the the tech debt and cost that comes from just

piling up too many like SAS Point Solutions too quickly like AI or otherwise yeah I I can’t agree more it’s

uh it’s looking at the need and finding the right solution for the need uh do

experiment you know ju don’t don’t uh don’t say you know it’s too early I’m not going in there do experiment but you

know uh you can experiment with lower budgets at the beginning to see that it

does provide the value that answers that your needs uh but you know Go full on

only if you if you establish the uh you know the uh the fact that it it it

answers the needs that that you have yeah yeah and I mean I I’m probably jumping ahead to the next topic here

that uh Anna is queued up but like what does that mean practically speaking if if you’re in an IT organization like

I’ll take from personal example so you know I in the past year I’ve had our our legal teams ask about evaluating legal

AI tools that help with like looking at Contract data to process things quickly

procurement teams looking at AI tools to do things like help with the procurement process and and V like privacy and

security and legal data that come with all the paperwork involved in procurement marketing asking about AI

tools for Content Generation Um and like in all of these types of

scenarios the questions you have to ask is one like if we’re already if each of these teams is already using SAS tools

to fulfill these functions are those tools going to be bundling AI features already like is it there in the road map

or is this really a case where you need a separate tool um and two are those needs so distinct and unique that they

require a specialized tool or if you have entitlement to something like

Google Gemini or chat jpt like is the generalized AI product set good enough

to fulfill those without needing it integrated with your purpose Bill tool the third consideration that’s now

emerging which is super interesting is the device and Os makers are also starting to commoditize this like um

with the latest version of iOS um Apple has started this where uh things like text generation and summarization are

built in done locally and available for free um you can go into the notes app or email highlight text do summarize like

give it bullets tell it generate like that’s something you previously had to pay for or use a third party service

what happens when more and more of those functions move into the devices or osses and become available at no extra cost I

think it forces the commercial Solutions and the SAS solutions to either improve

and specialize or they’re going to die and a lot of them will consolidate or die in the vine because they just won’t

be able to justify their existence as an it leader you sort of have to skate where the puck is going and think if

this is the trend then 6 months or 12 months is buying another product that just focuses on this use case plus AI

really the right investment or should I wait it out a bit make sure I have good requirements and see like where the existing products and platforms are

going yeah yeah definitely and and maybe chiming in on this uh so what we see

today is that there’s so much AI capacity built in I think it’s it’s no

wonder that Nvidia today is uh I don’t know if today it’s the largest company or it was tomorrow yesterday but uh but

it’s definitely out there uh but you the reason is that you know there’s so much

AI capacity built in so the the cloud centers are full of h100 and h200 and

and and all those uh those specialized gpus uh that that AI runs on and and and

you can act as Ryan said you know it’s you can actually go to chat GPT and have

free AI which is which is amazing considering the amount of of investment that went into this this should have

been uh should have been costing us a lot of money um I’d say that it’s if if

if I go back again it’s kind of similar to uh uh to the Broadband Revolution so

you know there was like in the in the early 2000 there was a lot of uh of uh fiber optics cables laid in a lot of big

companies that uh that laid all those fiber infrastructure and they were thought to have a lot of money that to

make a lot of money at the end many of them went B bankrupt but the infrastructure stayed we have you know

like a very very big infrastructure of fiber today and it’s a lot due to that and I think it’s kind of a similar

situation and we can enjoy that as as customers you know it’s kind we have all this uh all this good for free um I

would also chiming on on the use case uh and and I’ll give an example uh from uh

from at Tera so so let’s say that you have a that you have a ticket and you need to answer to a ticket and then you

need a suggestion on on what to answer so at a tera we have uh we have this

this button that allows you to uh to create an auto reply which is nice uh but honestly you

know you can as Ryan said you can do the same you can just copy the ticket go to sh GPT provide the ticket and ask him to

draft uh to draft response maybe you have a better prom maybe a little less but basically you can do more or less

more or less the same and and uh and you have it uh for free um however what you

know Chad GPT cannot do is what we’re doing with our agent so one of the thing

the things that we do is we have uh when when you have a ticket we just go to our

agent which sits on the machine and we we uh conduct several health checks that

are associated with the complaint that came in in the ticket and then we can also uh the the the r AI engine can also

uh uh create uh an action to remediate the issue that you have and this is

something that you know closing the loop this whole loop with ch GPT uh this is

something that’s you know that’s that’s almost impossible in other means so having this kind of a complete solution

that’s definitely something that you need uh a a point solution to to do that

and not just use a chpt so that’s I think it’s a good example for this it’s the context that you get with

integrating the AI features as part of a broader set of data and access and integration that your your platforms

have that’s always the key yeah exactly exactly I’ll I’ll summarize it as smart

adoption versus early adoption um right news cases from where we started from um

thinking about um smart adoption smart hiring um looking you know at your kind

of day-to-day as it leaders within your organizations um can you maybe speak a

bit about you know the changes you’re seeing um in terms of helping it teams

your teams to uh adopt AI do you feel like it brings up challenges um in terms

of recruiting new Talent do you feel like AI is creating a whole new skill Gap out in the market um what are your

takes on that yeah I think it’s interesting if you’re in an engineering role then it absolutely um has created

the need to to develop like some AI native skills like around prompt engineering and like if we actually had

a whiz posted a blog post about the under the hood for the the natural

language text to security graph query function we released where we talk about

the prompt engineering and like our mix of zero shot and fuse shot learning how we use rag uh retrieval augmented uh

generation um and and the architecture of the solution and and why we took the approach we did so there’s definitely a

lot behind that and um I’d say that it’s become an interesting like new track for

engineering development and growth what’s I think really nice about um this

all is AI enabled products and technologies have sort of been developed out in the open and except ible to

Consumers from day one even before they were contemplated as Enterprise Solutions and so access is really broad

um and it’s a smart pattern for the platform developers too because if if you can hook and engage um practitioners

as individual consumers and get them using and playing with these Technologies then you’re also building

this like ecosystem of like future developers and people who bring that interest and experience to their job um

and of course that serves them well so so um in that regard I think the the the openness and availability of these tools

has has been great for people to learn these skills I think if you’re a consumer of a product where there’s an

AI enabled feature um the skill set that I’d say is not like a dedicated

specialization but is more of just like a a technology foundations type of thing

is just understanding where these systems tend to fall apart like recognizing where a hallucination might

happen or recognizing the type of problem where like a product with AI feature is probably not going to help

you out like knowing the rough edges you know I think about someone who like relies on text content generation and

doesn’t proofread it or someone who uses autogen code and then doesn’t check it

properly for correctness um and again people like in experienced roles will

learn to do that through mistake and trial and error but I I think from a user perspective that’s where you’ll probably see a lot of development over

time I agree with with Ryan and so you know the way I look at it is there there

are two types of uh of roles that that are associated with AI so one is

developing AI systems and you know this is this is the place where you do need the expertise around uh you know Rag and

prompt engineering and uh uh and sometimes even uh creating your your own

llms through open source or in other means and training them that’s uh that’s

kind of skill that that uh that that you need and you know I’m pretty sure that

there’s a lot of uh there’s a shortage in the market uh I haven’t checked it but I would say that uh that that’s

that’s probably the case uh and then on the other side there are the AI users uh

and for the AI users I would argue that uh the main uh the main barrier barrier is um

is the being um being AI default so what I

mean is um I’ll take you again to a story uh in the back I have a lot of a

lot of history lessons here um so uh when Google just came out uh it was not

natural for me to look everything in Google look up everything in Google it was It was kind of it took me a while to

get there uh to get to the recognition that by default I’m going to Google every time I have a question uh and and

by the way Google worked very hard so that this will the case but but it came in the end and I think that this is this

is what happens right now with AI uh it’s it’s in there people are using it

but most of the people are not going are not defaulting to AI Solutions and and I

and I think that once they do it’s it’s something that uh that will entail yet

later on the learning that Ryan talked about you know learning when the results are okay how to how to write the prompt

how to use the context window how to use your own context uh a lot of things that

that are kind they’re not complicated but they’re kind of you need to use the system in order uh to get uh to get to

know them um so so I would say that this is the main hurdle for hurdle for

adopting uh AI right now I’ll give another another example from uh from my

organization so um um the product managers at at atera one of the things

that we do is we write uh it’s called Product backlog item is kind of a a description of what we want the uh uh

the dev team to uh to develop and and you know there’s a lot of uh there’s a lot of field in there that uh that we

need to address such as security and uh and and and data events and a lot of uh

a lot of things that need to be addressed in order to uh to provide a good productized feature um and and you

know what you know when Chach PT came out a little bit afterwards I told my guys luk you know let’s let’s build a

skeleton with uh with uh with AI That’s uh you know that that should be pretty easy and it was very very hard to

convince them to do that and and actually uh you know only when when we wrote uh you know the the right prompt

and show the result this is something that that created adoption and then you know people tended to experiment with us

and and change the prompt according to their needs and and and start start working with us so it does need uh a

little bit of nudge uh from users to to start using it and and default to that

uh to that solution but I think you know we’re going to get there um great thank you um both I want

to go back maybe to something uh you mentioned earlier earlier that kind of relat to this um um both hiring and

maybe people’s sentiment towards AI um tal you mentioned you know some

capabilities uh even with AA include potentially not having any technician um

interfere um right with a ticket solving process so I’m wondering um are you

still feeling people are nervous about AI replacing them taking away their jobs

um maybe especially with you know Junior Tech roles um would love to hear from

both of you I I I would say a couple of things on on this um so people are nervous for

sure um I don’t think they have a reason to be but they are nervous uh if we look

at the uh at uh the the it domain uh today again that’s it’s the same uh the

same McKenzie research that I’ve seen I’ve seen earlier uh it says that you know they research about you know a

million jobs or something I don’t remember how many but what they saw is that the number of skilled people is uh

is about onethird of the open positions in that field in the in the it field

that means that you have about you know uh two-thirds to fill in uh with the

with the open positions which is which is a lot and and I think most of you out there you probably feel it it’s very

very hard to recruit people to those to those IT jobs uh you know wages are

rising as well as a result which is which is natural and you know it’s it’s

it’s kind of it’s kind of difficult so so I would say that you know if if we do

manage to uh to reduce the need for human uh from for human intervention in

in tickets for example that’s probably going to uh alleviate the uh you know

the uh the the severe or uh the severe shortage that we have today with with it

persons um I think I think that’s one and and the second uh you know the

second issue is is uh again history lessons I think history taught us that

every time that we’ve introduced uh a new uh a new prod productivity gain um

then people didn’t lost didn’t lose their jobs actually what happened is that people went and did other jobs that

were uh that that they didn’t have time to go to uh earlier and that increased

the overall productivity so for it people again you probably feel that uh

many times when we talk to uh to it professionals they say that they drown in the dayto day they need to answer

tickets they need to uh to solve to solve problems they’re very very reactive and they don’t want to be

reactive they want to be they want to be proactive uh but they don’t have the time for this uh I would argue that uh

that with AI Solutions that’s going to be the case so more time invested in in

uh uh you know avoiding security issues and and handling infrastructure and

being very very preemptive uh rather than being reactive uh as they are today

that’s the way I see it yeah um really really well well put T

I think um it certainly varies by industry and rooll like I do think that we’ve seen already effects in certain

markets around content creation um uh content moderation other areas where um

the cost uh considerations of uh more entry-level positions versus the good

enough output you can get from uh llm enabled products has actually started to

have some effect um I think in Technology and Engineering it certainly has been more insulated and that is

where I think as T said there’s um such a skill shortage across all tiers both

entry level through more experience that um even in cases where there are efficiency gains there’s so many other

areas of need and opportunity that it’s more of a shift of where resources are put rather than an outright replacement

of of people with with systems yeah I I would also add that uh that

that uh it’s true that there will be some um some uh kind of professions that

uh I don’t know maybe they’ll disappear maybe they will uh kind of uh dwindle a

little bit uh however those people will probably do other things um what of the

uh uh you know one of uh of uh the ways to look at it is and I heard this from a

from a friend of mine uh he said that you know in in the content world and and

and you know image generation uh we went from being uh creators to being curators

so so you do need someone to curate so so you go to Mid journey and you create

those images you do need to select the Right image right you do need to change the prompt a little bit so so it’s it

did not disappear the results are better they’re faster you can iterate faster but you do need that person to do that

that’s right um I’m I’m watching the chat I

just wanted to shout out Brent who said he’s been using co-pilot that’s how you

know you amplify your resources uh without hurting anyone he’s been using

co-pilot to create cleanup scripts and many others so um if anyone’s here that’s been you know giving co-pilot a

try please uh let us know uh what you’re doing with it we’d love to hear from you

um and with copilot I also saw a question coming in in the Q&A I’m I’m reminding everybody um any questions you

have you can add them to our Q&A tab one of the questions had to do with hallucinations Ryan you touched a bit on

it um but maybe um let’s kind of move into this conversation about limitations

and security risks um because obviously everybody uh cares and and worries about

that so um for Security leaders um managing Tech within companies how do

you handle um third- party vendor risks when you have you know team members

coming with these different tools that they may use at home or bringing them to work um how do you go about that yeah

it’s a it’s a great question I’ll say this like very candidly third-party risk and vendor risk management is probably

one of the hardest problems to solve for it in Enterprise Security today um and

uh unfortunately a lot of the solutions and approaches over the last decade or two have led to a lot of paperwork and

um unnecessary friction that provides very little value some of this is rooted in well-intentioned compliance and legal

work that just goes off the rails but I’ll give an example um a year or so ago

we introduced um uh in our product documentation which is in product docs a

search feature that was powered by AI it was actually provided by the third party that we use for our our doc solution um

and the moment that that was in the release notes as a feature to search documentation um we had compliance teams

from Enterprise customers sending two to 300 questions about AI safety and risk

with questions about like what’s the risk of physical harm and human harm and bias and this was all triggered because

for some organizations they’ve built up the scaffolding to say like AI usage creates Risk Technology has AI therefore

the way we address this is we ask the vendor 300 questions and then it goes in some sort of like of action and

Milestones in some giant system and um the same approach has by the way been used for a lot of security work and it’s

hard because as a consumer you can’t introspect and monitor every single one of your vendors and their environments

um and so I I think the the downside of that is it it creates friction without

benefit like some friction in security is good but when you’re as a consumer of a new technology like the questions you

should be asking yourself about AI risk need to be rooted in the same questions you should already be asking about data

and access management like you’re handing your data to another vendor that vendor in might have their own

subprocessors or vendors do you understand who it’s being shared with do you understand the effective permissions

they have to your data versus what they claim they will use do you understand the life cycle of the data do you

understand the security controls that are in your side of the shared responsibility model and if you’re

capturing those things you can actually build CR IIA and ask the hard AI safety

and questions where it’s actually Justified um and not take this sort of like simplified naive approach where you

just cluding everyone to bludging everyone to death with with huge questionnaires so everyone’s working

through this now um it’s it’s a real challenge I will say and um I think organizations are also struggling with

the fact that there are no standards for assessing AI enabled products and

solutions in the context of risks like there’s no like uniform third party risk

assessment standard I can hang my hat on and say like we’re good this was reviewed and vouched for by this entity

that’s operating with a standard therefore like this should give you some level of assurance that has to be

developed um until it is a lot of orgs are going to be spending a lot of Cycles trying to figure this

out wow I couldn’t agree more uh I think um you know I think uh

the U the compliances around this and and by the way to hear that from uh from

aiso is a breath of fresh air um it’s uh

it’s crazy and I think I think one of the one of the reasons is that uh you know compliance like the regulator that

created the compliances uh instead of uh going to general rules that everyone

needs to apply to to uh to adhere to by law uh you know and and and and say

these are the rules uh they went on to create compliances that requir require

you uh to document everything and to research every vendor that you have and

and go through you know so many uh time wasting and energy wasting and and money

wasting uh uh areas uh maybe a little bit on on how we handled this in in in

in atera uh where where we went so we we went in in in two directions one

is you know we partnered with h with Microsoft and we said look you know we know Microsoft is the is the Enterprise

Queen right you know they have a lot of a lot of uh a lot of people invested in

in compliance so we’ll partner with them and you know we’ll go by their rules and and uh you know and we’re going to keep

it safe and and second is that we’ve actually uh we’re not using Genna

uh as uh as as something that’s free uh to Ram it’s kind of limited in what it

can do so it has several actions that it can do one is health checks he can run

health checks and those health checks were written by a tera and they can be applied so they’re safe there they’re

completely safe and then a set of actions that are uh a lot of actions are

safe for the end users and some actions that are safe for the technician so

so the AI cannot go wild and start you know start uh doing stuff on your on

your computer it’s kind of limited to uh to to what it can do and to what we allow it to do and we only allow it to

do automatically safe procedures and and allow the technicians to approve others

that are that need a technician to approve so it’s a bit on how we took it

what what’s also interesting is in the earliest days of like chat GPT I think there was a lot of hyperbolic media

coverage about the risk of um data leakage of like the system somehow being

trained on one question from one user and then like sending sensitive data from that to another user and the

reality is if you actually look at like evidence-based sources like yes

academically like leakage is a risk but the bigger risk is actually a number of other things so for instance in

multi-tenant systems prompt history and how that’s segregated like the the risk of model leakage leading to sensitive

data being disclosed in a response to another user is far lower than prompt leakage because the prompts often

contain a lot of the sensitive information and if that’s being logged in a multi-tenant system like protecting

that’s interesting access is interesting if the system is an agent and has access to data and can like violate trust

boundaries um those are the areas that like people should be focusing on um a lot of times like the the model leakage

threat because of again the early days and understandings of of how these systems work end up being where um a lot

of focus is preoccupied on and it’s it’s not where people should start yeah

definitely yes I think we’re we’re seeing a comment from Eddie kin speaking just um on that saying the Big Challenge

uh is how Ai and llms protect company data and the reason they’re not using CAD GPT um at the company is protecting

users from leakage but Ryan you just chimed in on that and that’s super interesting and there’s another question

um in our um Q&A tab um asking you um as

with uh ciso and CIO how do you explain um to your um employees and colleagues

how to use geni um from that perspective exactly and as you can tell I’m moving

um towards the Q&A tab as we have a few more minutes to kind of go over all of your questions guys so so I look at this

as an extension of what you should already be working with your employees on which is acceptable use for for third

party services like regardless if they’re free or for pay um because the

reality is it doesn’t matter if a SAS product that your employees decide to use for company business is AI enabled

or not if I have employees that are using company data with an unsanctioned sas platform or solution that’s a data

security and privacy risk that I need to manage regardless and so so what we’ve done is tried to extend the conversation

in two ways one is to make sure users understand that part of acceptable use and part of what we monitor for as part

of our governance is we provide a set of tools that are authorized and reviewed and managed and that’s what we provide

for you to do your work if you need something that’s not being covered by those have a process for that the flip

side that is it leaders I think is the responsibility to um uh be proactive in

finding those use cas and like spotting where existing products are not filling the need because if you don’t do that

then people will work around you people will use unmanaged devices where they can they will use unsanctioned Services

where they can get away with it um and so like if you don’t work with people and give them some good paved Road

defaults that you can manage um that’s the the failure mode they will just try to bypass you at every turn so I think

that’s the shared responsibility between it leaders um and and employees

I’m I’m I’m not going to answer this I’m not a CEO so I would say sure you have for using a

ey though within your te I just I just urge my my people to use AI to the

greatest extent that they can and rely on the sister to limit them when it’s not

possible um I’ll maybe I’ll jump back into a bit of the conversation about productivity because I’m seeing a a

great question from Lars um asking how can companies ensure that the implementation of aite tech not only

leads to short-term productivity gains but also promotes long-term innovation without disrupting existing

processes maybe I I’ll have a shot here so so I think we touched it earlier

and and and I think as I said um one of the things that you to drisk the

implementation of AI and one of the ways to drisk AI is to have um you know Ryan

mentioned it look at your needs see if there’s something that can answer your need and then maybe look at the business

model and see if that business model is per success so so that you have nothing

to lose so if it succeeds then then you’ve achieved your needs and if it if it does not exceed does not achieve that

then you pay nothing uh I think that would be that would be good Bal balance

you start to see these kind of uh uh companies and these kind of solutions out in the market um and and and I think

that guarantees that you know that if there is productivity uh gains to be had

you’ll have it and if there isn’t at least you didn’t pay

anything great I want to I want to thank uh Brian for taking on the question

about a ice certificate from our Q&A tab um I love it when

people help each other in our chat um maybe you guys want to add to Hudson’s

question are there any certificates available in AI for someone already working in it uh or maybe any other uh

courses you would recommend um I’m not actually familiar with any

certification uh available I I’ll just say that uh you know from from the usage

perspective Ive just just try it just try to do it it’s not very

complicated uh and and you gain a lot by having experience Hands-On uh but you know from compliance

and other and other measures maybe Ryan has a as a more detailed response to this not for certifications I mean I’d

agree that like with with open models like llama um available it’s it’s easier

than ever to get started an experiment on your own time and I think um with security and it alike I’ve seen a

lot of great engineers and practitioners just build up that experience like as as people have done with their home labs in

the um for for decades now um to uh to come up with some projects and problem

ideas and experiment and learn learn by doing I think that’s always the best way and you can use AI to to learn how

to do it as well that’s right ask for some ideas um okay great um if um if that’s

okay with everyone we’re going to go to our final question um I want to talk a bit about the future um what do you guys

look forward or what what what can you share with others that that we can expect um in the future in terms of

where productivity and additional gains um when it comes to integrating

AI um so I think as as I said earlier

I’m a big believer in in a shift inct activity that will be uh provided by

using generative AI uh but as I said earlier it’s very hard to know when this

is going to come it’s uh you know it might be that uh we’re in uh you know

we’re entering the trough of disillusionment it actually makes sense that we are uh but you know like the uh

you know the uh uh the inclination of the curve is yet to be seen how fast is

it going to be how fast we going to reach maturity it’s it’s very very hard

to know uh so it might be that you know we’ll start seeing uh uh productivity

gains in uh in a couple of months from now it could be years from now it’s very very hard to know yeah I I would agree

like there’s some experts that feel like the current approaches are going to hit a wall and that we won’t be able to

overcome some of the things needed for like the true vision of uh fully autonomous agents and reasoning to reach

what we think it could um but that may also not prove to be the case so um I’ll

say in the meantime what I’m looking forward to is more and more AI technology features to sort of Fade Into

the background of how things just work um again taking the analogy from mobile

um I was reflecting on the fact that with the latest iOS update you have things like summarization of your text

messages and notifications as a built-in feature and as a user you don’t really think like that’s I mean you they

advertise it as an AI feature but it just happens and I think more and more of how these systems and technologies

will get integrated will just fade into the background like that and that’s I think the best outcome for a technology

that’s when you know if it’s reached a point of maturity where the it it is not the selling point on its own it is the

selling point is the outcomes it enables and the user experience um and so I look forward to seeing more and more of that

because I think again centering on the value and the outcomes and the user experience is how you get great technology in security and it in any

domain thank you thank you so much um this is this is bringing us up to a

minute um I want to say thank you to everybody that joined if we missed any

questions uh do not worry we will go over them um and you’ll be able to get

answers to all the unanswered questions in our community um through all the other channels our social media channel

channels um thank you tal thank you Ryan this was super insightful um and I hope

everybody had a great time thank you so much I had a great time thanks Ryan

thank you take care