HUMANS IN THE LOOP; ROBOTS ON THE TEAM- PART 2

Browse more episodes here

READ THE FULL TRANSCRIPT

[SYRIL SMITH GARSON]

As we think about access to data and how our relationship with data changes, is that access to unstructured data at scale, being able to digest it a lot quicker, being able to pull that information into some sort of investment thesis? It's almost this idea moving towards this idea of quantamental, or quant, for everyone, not just the systematic teams.

But maybe how do we think about data more broadly within this space and access to data?

[ADRIAN CROCKETT]

So obviously, I think, within financial services, we have two different sets of data, what we can get from the web and then also what we pay for from premium providers. So, the premium providers are going to be very focused on how do they actually protect their brands and their reputations, and also their business model. So, making sure that we're actually working with those parties and that we've got the differentiation between what has been scraped from the web versus data that's coming from premium providers. And the other thing that, for me, is really interesting here, is when we're thinking about data, is really thinking about the breadth of the data. Not just the quality of the data, but also the breadth of the data. Sometimes we just think about quality of analysis. How would the human have done it versus what the machine can actually do?

But there are many different things that just aren't happening in financial services. With the breadth of unstructured data in particular, you just can't actually keep up with it.

[SYRIL SMITH GARSON]

No, there's no way.

[ADRIAN CROCKETT]

That's not a task that a human can do. So, the results are going to be significantly better, not because a human versus machine output, but because the human wasn't doing it in the first place. But if the human wasn't doing it in the first place, it's not really just augmentation.

[SYRIL SMITH GARSON]

Yeah, absolutely.

[ADRIAN CROCKETT]

It's actually setting up a new set of analysis altogether.

[SYRIL SMITH GARSON]

That's really interesting. That goes back to our, do you augment versus transform workflows, right? And this feels a very clear answer to that. Transforming of the workflow is really having that Copilot be part of the process. You briefly touched on web-scrape data and provider data. I'll maybe give you a middle bucket of that, which is alternative data, and how we can start to use alternative. And I mean that with a lowercase a, meaning different types of sources, not private markets data.

How do we start to think about what that also unlocks, not just for the most sophisticated end of investors, but for really anyone, where how can we start to use that?

[ADRIAN CROCKETT]

I think that question, we need to also be trying to address that from a multimodal perspective. So, depending who you are, you could either take the word data—many people in finance might think about that as structured data. These are the fundamentals, market prices, and such forth. But we also know that we have a wealth of unstructured data, whether that's news, analyst reports, there are a variety of other sources that we should be incorporating into investment decisions.

But we also, for the last 15 or so years, have been taking photos from a satellite over the top of big-box car parks to try and work out how many people are actually going there. So, what's the next generation of that? Well, the next generation of that might be, well, can the satellite actually infer the types of cars? There's no reason we can't do that. We're already doing that in the insurance space.

[ADRIAN CROCKETT]

So, in insurance, you have a car crash. Now you take a photo of the car. The AI understands what the car is, understands what the estimate of the damage is, books you in, et cetera, et cetera.

[SYRIL SMITH GARSON]

Absolutely.

[ADRIAN CROCKETT]

So, let's go back to the big-box retailer. Now, the satellite image actually takes the photo, actually does an assessment of the different brands of cars, and can now do a time series of the average expense of the car that went to that big-box retailer. So, can I create an investment thesis out of that? That seems like something I probably want to run some data on, right?

So, I think actually going through these and saying, listen, we've got a lot of different data that, historically, wasn't available to us, that we can now extract the information out of it so much better and form better investment hypotheses and create different models is going to be something that we're going to see significant movement in.

[SYRIL SMITH GARSON]

And we talked about multimodal. What is multimodal?

[ADRIAN CROCKETT]

So, we're both communicating quite a bit. We are talking at the moment, we're putting off data. But we both do something in common. We both speak with our hands. We're both very expressive in our face. We're actually communicating not just through the audio, but through visual elements. So multimodal is really about how can we actually take the combination of text, audio, and also visual elements and actually extract data from that. You see these instances where you're actually going, listen, I'm actually going through and asking a natural language question.

So, people who have used something like DALL-E or Microsoft Designer, I'm actually typing in, my teen daughter does this all the time, right, types in a description and then gets the image. So that's an example of text to visual.

Then we flip that around. We go, now actually take an equity analyst report, for argument's sake. Weirdly, we might not think about it as a design chart, but there's a lot of charts and tables in there. So now what I need to do is I need to take that and extract the data from the table and put that into words. I might want to do that because an equity research report is actually a combination of unstructured data and then charts and tables. If I want to provide a text summarization over that, I don't want to miss out on the charts and tables. That would be a big mistake because, as we spoke about a while ago, in finance, we use charts and tables to express data. So, we want to make sure that we're capturing all of those elements.

I think within finance at the moment, the predominant elements are a lot of natural language or text, visuals are becoming really the focus area. As you can imagine, everyone is super excited about audio. Think about an earnings report or something of that nature. Speed is actually a critical element there. You know, I like to think about a company talking about an expansion plan in Asia, for argument's sake.

Well, when the CFO actually talks about that in the earnings transcript, I want that extracted from the audio file. But then I actually want that tagged and then I want to see a time series of that. And then I want to combine that with correlations with what their share price did in real time.

[SYRIL SMITH GARSON]

Well, I was going to add the sentiment over top of it or compare that to the actual earnings statement. Are they saying the same thing? I think it unlocks those capabilities as well.

[ADRIAN CROCKETT]

100%.

[SYRIL SMITH GARSON]

We talked a little bit about this idea of now we've got everyone coming up with really fantastic ideas and need us to be able to design for, think about, componentize all the pieces. How does the way that we design and build and think about products now have to change?

[ADRIAN CROCKETT]

So, I think this is really the fundamental question. The problem that we're actually faced with here is actually how do you build the foundations so that you can have all those plug-ins and skills actually interacting with each other so that we actually have a multiplier effect and, ultimately, that each different person on a trading desk or at BlackRock has actually got their own copilots. We need to facilitate hyper personalization.

But the problem as a product person, I think, is significantly more about how do we actually create those multiplier effects so that we do get to that position where every person can lean on a copilot to help them get to done and do their job.

[SYRIL SMITH GARSON]

Well, and it's really interesting where we are now, we're very focused on how do we build this platform-- to your point-- and a set of common services and capabilities that underpin all of these different copilot experiences. There's two things that maybe I'll pull on there that you said, which is this idea of everyone has to have their own copilot.

I think right now, the mindset we're seeing is everyone has to have their own interface. We're shifting that to say, you don't necessarily need a net new user interface, right? You do need to work towards building your own copilot. That's fine. So, I think there's an interesting conversation to have there of how does that transform from I need a UI to I need a copilot. We're working on that transition.

But then the second thing we keep getting challenged with is, OK, now everyone on the investment team has their own copilot, and it does slightly different things, right? And maybe it needs to, right, for the investment process to be unique, for us to be able to generate alpha, there are use cases where these copilots have to be different. But then navigating across all of these copilots, right? And how do we think about this world, as-- you started this conversation with, where are we going to have the copilot that sort of rules them all, right? How are all these going to plug into?

How do we move from just these one-off interfaces, one-off MicroApps to now plug-ins and capabilities. And how do we think about Aladdin, right, as also an ecosystem, where we can now have plug-ins and capabilities across all of our own applications that allow you to tap into skill sets. That it can act as this connective tissue across the entire platform. I think, in a lot of ways, what we're doing with Microsoft Teams today in BlackRock is that. We're saying, how do we have each team be able to build their own plug-in or capability into the product?

[ADRIAN CROCKETT]

I think the other interesting thing, just because you've used a couple of phrases which are really about the copilot stack in some respect is, traditionally, we always thought about the product as being the interface that people went into. If I'm talking to an engineer, obviously, they're thinking about the APIs and such forth. But there was one level of product that we often focus on. The interesting thing for a firm like BlackRock is I can imagine one of your customers experiencing your copilots, but I can also imagine them actually wanting to use your orchestrator. I can imagine them wanting to use your plug-ins. I can imagine them asking the question, hey, can we continue to use the BlackRock copilot, but we actually want to inject our own plug-ins and give priority to that. The flip side might be is they might say that they've actually got a custom copilot and they want your plug-ins exposed there.

[SYRIL SMITH GARSON]

100%.

[ADRIAN CROCKETT]

So, I think there's a myriad of different ways that people are going to come at this, and really architecting for that flexibility. But this is a new paradigm and we're still in the very early stages. And a lot of customers in financial services also do actually have very large technology teams themselves.

[SYRIL SMITH GARSON]

Yeah, absolutely.

[ADRIAN CROCKETT]

So that also changes the dynamics about whether they want to actually be able to custom copilot, whether they want to leverage yours, whether they want to leverage your plug-ins in their custom copilot, and such forth.

[SYRIL SMITH GARSON]

And we are hearing that from our clients. I think that really resonates. The other piece that you said that resonated with me very acutely, as we think about product management, is I'm spending less of my time on, how do I think the experience should be, the capabilities, the features that need to be for the sort of bespoke use cases? And I'm spending a lot of my time thinking about what are those services, capabilities, access points, platforms, that or that sort of LEGO piece or that plug-and-play that you're talking about as well.

[ADRIAN CROCKETT]

So, there's a lot of philosophical discussions. And as I alluded to before, it's not a technology discussion. The technology or product side of this is one portion of this. This is as much about organizational design as it is about technology. I think the other set of discussions that are very prevalent at the moment is, how do I think about that platform layer or the services layer to actually optimize the return on investment?

And it's great that we're sitting at BlackRock because I can use the word optionality, right? This is about how are you being a product person that actually facilitates optionality for the unknown future? There are paths that we can see spanning out by actually facilitating that optionality and recognizing that a lot of our customers are very strong and have very specific views that we can also facilitate all of those.

[SYRIL SMITH GARSON]

Yeah, I love that. We had briefly talked about this idea of you open up your computer, your financial dashboard in the future and you see that you've got a search bar. And that's where you start with. Where do you think we are going to go with user experience beyond that? What is the next action step? What happens after that?

[ADRIAN CROCKETT]

So, I think it's an interesting question from the perspective of the way that you phrased it it’s still a pull event. I'm going into my laptop. I'm going into a search bar. The mental model that I have is very much around copilots, is they're like a fantastic intern, who never sleeps and never eats, and also decides to take the long-term job offer, I should say, right?

If you had a person like that, what would really happen? What is the ideal sense? The ideal sense is when you actually wake up, pick up your phone, and it's actually given you an indication of what's important for you today. It's actually helping prepare. So, I think the paradigm, for me, is very much, how do we actually evolve this copilot process from a pull event to actually being pull and push and also representative of once-off tasks or low-frequency tasks versus repetitive tasks. How do we actually expand that?

And I think when we're thinking about that future, the experiences are going to evolve quite significantly. So, I do really want to wake up and have someone help me out and say, listen, there's only so many hours in the day. 

These are the things that you have to do. These are the things that are actually going to move the needle. And when we're thinking about what the implications are on the experiences there-- and I think it is actually thinking about this as an experience rather than just the UX/UI. We're both product people in financial services, but your job is very different to mine. Mine is very different to yours.

So, there's not going to be a copilot that can actually describe all those things, given what our calendars also look like, the calls and such forth. So the experience there on the generative UX side of things is going to be very different. And that's why it's going to have to be very much dynamic. Because that is really the hyper personalization element, taking into account all the data that's in my system and then represents about what I have to do versus what you have to do.

[SYRIL SMITH GARSON]

I really like this idea of the generative experience. I think there's another implicit thing that you had said, which is it understands the intent of what you're trying to do as well. It's that nuance of it understands your calendar versus mine, but it also understands your behaviors, your patterns, your preferences, right? It remembers conversations that you've had before. I think over time, that ability to intuit what you're trying to do, I think will be really interesting as well.

[ADRIAN CROCKETT]

So, I think in the very early days of generative AI, everyone jumped out. We were working together. We built a whole series of copilots. And the approach that we went through was very much about how do we actually build that one POC. Now how do we build another POC? And in many respects, every major firm in financial service has gone through that process of building those POCs.

[SYRIL SMITH GARSON]

Absolutely.

[ADRIAN CROCKETT]

In the background, what's also been happening is the number of use cases that people are actually asking for has doubled, then doubled, and then doubled, and doubled, and doubled again. So, then people like us have to start thinking slightly different about what the actual problem space is. So, this is where the thought process has really evolved to the platform and services level. And that's really, in some respects, it's like a Taylorism movement, right?

You're actually going through and rather than actually building a car once off, like as a bespoke car, we're actually saying, no, we're actually going to need to get ready to actually operationalize this.

And that is actually setting up the factory floor. And now so what we're actually seeing is firms who are actually investing in those factory floors, the speed at which they're actually being able to get the Copilots out, or the plug-ins and skills also, is just growing.

[SYRIL SMITH GARSON]

I love this idea. And I resonate so much with how we're thinking about it, of stepping back and building the factory floor first. That's very much where we are now. And I think the other thing that maybe I would add to that, that I feel like I took for granted when we started this, was getting to the POC –it's actually kind of easy. We have no shortage of POCs across the firm.

However, it's really getting it beyond the POC, making sure that it is consistently accurate. I think one of the things that's been really fascinating to me is there's no scale 1 to 10. This is an 8. Like, there's no easy way to evaluate the responses. There's no universal way to do that yet.

I think many people are working on that evaluation framework, but there's no easy way. And there's no easy way to say, OK, well, this product, the responses that you're getting back, we've agreed that it's a 6 out of 10. That's good enough to go live. It's sort of very subjective right now. And so, I think that's the other piece of this that's been really interesting for me is that POC is easy. getting past the POC has been a much bigger challenge.

[ADRIAN CROCKETT]

Yep, and I think the other thing that's interesting is, when I think across all of my customers and partners that I work with, there is not one central AI team at any of those firms. So, what is the unified copilot strategy that the firm is actually going to implement rather than actually sitting there and saying, Team A has done this, Team B has done that? Because you can go forward, and there's many firms who I work with who have literally got 1,000 POCs up and running. But if you have 1,000 POCs and you continue to actually productionize on the basis of all those POCs, unless there was a unified framework, it's going to be very expensive to actually maintain, it's not going to be scalable. So, I think what we're going to see is a lot of firms create their own custom copilots. Now, in order to do that, you need to actually create the plug-ins. There's no copilot without plug-ins.

[SYRIL SMITH GARSON]

Right. You can't do anything.

[ADRIAN CROCKETT]

If you were to ask me the question, in three years time, does someone go to M365 Copilot if they're in a Microsoft environment to ask an Aladdin question or do they go to an Aladdin Copilot to ask an Aladdin question?

That is going to be very, very persona specific, or down to the individual. But going back to the optionality that we were talking about before, our job is to facilitate both of those people. Because as long as Aladdin is providing that person with value and also that you can actually get the providence and such forth, so if the answer's coming back in M365 Copilot, you're going to want it to be very clear that this was powered by Aladdin. You're not going to want your data co-mingled with anyone else, right? So, there's a bunch of different stipulations there. But as long as you're providing that insight and the person is actually referring the value of that exercise back to BlackRock, I think you're relatively agnostic to that.