DevOps Speakeasy with Sean M Phillips @SeanMiPhillips at #Devnexus 2023 (Part 1)

so hi we’re here at the Nexus and I’m

sitting down with Sean Phillips and he’s

going to talk about us uh the

the topic of his Keynote so tell me

we’ve already had this conversation like

starting this conversation and I was I

was saying like I know you since some

years ago and before you were literally

working for NASA so if a person

my

my perception yeah but you were

you yeah I mean before I worked on that

a lot a lot of NASA ground systems uh I

was I was a software engineer that

worked with the mission design people

worked with the propulsion people worked

with you know the the science people and

made sure that their ground systems

their even their onboard software worked

properly right

um and that was good stuff I did a lot

of deep space stuff and I

I guess I got a little bored

and so I know it sounds like really

really like how could you say that but

like I just after doing it for eight

eight or nine years I

I decided that I wanted to do something

different and I but I still wanted to do

visualization work so I changed my

position I now work for the Johns

Hopkins University Applied Physics

laboratory okay okay uh APL for short

and

um we do in my group we do a lot of work

with data and Analysis and machine

learning okay so one of the things I do

is make a lot of

custom high performance 3D

visualizations to support the various

data scientists trying to get their work

done by providing them with interactive

visual tools that let them analyze not

only the data but the models and how the

models perform and that sort of thing

in the keynote I’m pointing at the at

the amphitheater is is going to be this

it’s this concept of

real life

neural networks I.E your brain

which have inspired artificial

intelligence neural networks AI yeah

yeah okay

and then the tools that have been

developed to analyze AI then being

turned around and reapplied back to

actual neural net biological neural

networks that that sounds like and then

incredible well I appreciate it uh

that’s and it and really the

presentation is just that cycle

but captured as a three-dimensional

visualization tool that I built before

that okay and does it match because

sometimes I have read that uh how do we

think our brain works like how do we

model the different layers and the

networks that we create uh it was an

abstraction that maybe it was not like

100 percent right fated but now that you

you close the circle you close the cycle

you’re getting more

I cannot honestly even imagine them out

of feedback onto on that information

well it is a lot and let’s be clear we

we as a as a research Community have

only just scratched the surface

and I want to be very clear

I’m a software engineer

my specialty is moving pixels around

and and I do it really fast and I’m

really good at that but my job is to

make sure that the people who are the

actual neuroscientists can actually do

the research as effectively as possible

so they’re the ones that are doing this

amazing research and and so I I want to

make sure that credit is it goes out

right yeah

understand the domain I mean yeah

it’s true

it’s amazing I tell people I do believe

this though when people say well how

hard is software engineering I tell

people it’s kind of the hardest it’s

pretty much the hardest job that there

is in terms of like technical careers

because you have to not only be a master

at your field but then you have to

become a master at somebody else’s field

every six months every every time your

project changes you have to now learn

somebody else’s job and you have to do

it well enough you have to learn it

enough so that you can then make it easy

for them exactly but it it took them

years to get to where they’re at and so

when you’re constantly learning new

career Fields every other year or

whatever I mean that there’s a lot of it

just becomes that’s a lot of pressure

that’s a lot of effort yet we just do it

and that’s

the abstract yeah exactly we need to

learn how

yes because our where our work is about

changing so much domains technology

strategies paradigms architecture

Everything Changes right and this is one

yeah you’re right

about the keynote about

well

of course for you for you okay all right

thank you very much

the inspiration for this software came

from

a project that my work my my the APL

brand which was called um Neo Neo stands

for neurally enhanced operator okay now

I I can’t share all the details because

of the sensitivity reasons but we

actually worked with several you know

other universities and we had a human

subject that had an actual brain

computer interface surgically implanted

inside of his brain wow

yeah wow uh and and and he would come in

in the morning and they would they would

plug in

actual electrodes into his brain with

all these pins like think like a

like a like a VGA cable or whatever

except that instead of 15 pins there’s

300 pins or a thousand pins and boom

boom boom and then my job was to make a

visualization that was interactive that

he would they would literally have in

front of him and he would sit there

plugged in and not only manipulate

things on the screen with his brain he

would receive information and then have

to make choices based on what he

received and then the neuroscientists

who not only did the surgery but also

understood how these things work they

had all kinds of different experiments

to try to test how fast he could react

how how much of a signal could he

receive what could he do with signals

coming in and also doing other normal

motor activity at the same time and and

these are things that nobody knew the

answer to

figure out how our actual Network neural

networks are connected right I think it

was in the recently in the news that it

was we were able actually to map the

connections from a larvae a larvae or

something like that something much

simpler yeah yes yeah so yeah to be fair

the human brain is just like it’s

incredibly complex and so they’re even

with these electrodes they’re only

looking at a very small subsection of

neurons and even that was

grossly complex to the point where I

think we needed very special

um Dimension reduction type map to even

decode it and what we found here’s

here’s the sneak preview is that

you know that movie Pacific Rim where

they’re in the big robots and they’re

thoroughly connected

but then they drift away they slide away

in and everything gets screwed up that’s

exactly what was happening they would

they would train their decoders on in

the morning on how his brain activity

was operating that that morning and then

the next day his brain had slid from the

decoded state so they need to like

refresh they had to retrain every single

morning but that training process took

over an hour and they only had them for

five or six hours a day so when you’re

spending

uh you’re spending almost 20 30 of your

time just retraining that’s that’s a

loss that’s a lot of resources that

you’re you’re losing out on right

because it’s just this person’s a one of

a kind yeah personally there’s nobody

else in the world like him so

we we accepted that as part of the the

challenge but then near the at the end

of all that I I was reading some papers

and I thought no there’s gonna be we got

to get better at this but we don’t know

why we’re not getting better so when you

don’t know how to solve a problem yeah

one of the best ways to understand that

explore is through visualization so I’m

native 3D viz which showed how when when

the when the human subject did a certain

activity made a motion with their hand

that we’d asked them to do how did the

brain react yeah

and what we found and we applied some

math to it that I barely understand like

I know the 3D stuff but like the actual

math is like Voodoo to me but and what

it showed was it we were actually able

to visualize in 3D using Java effects

how how how the trajectories of this of

the neural Pathways firing how

essentially how they were changing from

day to day we found that it was actually

a very similar shape it’s just it’s just

it’s just rotated in space and so the

reason that decoders were failing

because they were being trained on an

arc in in 3D space effect well

hyperdimensional space that maybe look

like this but then day one it’s like

this in day five it’s like this and Day

Day 63 it’s over here but it’s the same

shape yeah and so what they really

needed to do was was it yeah transform

it into and then look at it as a pattern

not as a specific set of numbers but by

then the pro the program had completed I

was doing that as an internal research

and development thing so now we had this

like kind of like one-off visual

capability that that proved that you

could look at hyperdimensional data

effectively in 3D

so we kind of step back and we’re like

well what else could we do with this and

somebody who was working on artificial

intelligence models uh you know overhead

imagery recognizing objects said

well can you do that for my

hyperdimensional data because that’s

that’s when AI model is that

it’s the same thing

yeah right vectors of vectors make

matrices of a floating point and

it worked like it’s just like the same

visuals really worked well and we just

so I just kept building more and more

features to help out the AI people

and then a year had gone by one of the

other Neuroscience people had come back

to me and talked to me and said hey

remember that visual that you made on

irad

well we got this new research brain

computer interface research program what

if I gave you some of the data we

collected could you plot it for me

but by then I built so many other tools

for AI that we had not tried any of

those new tools on this old stuff and

that’s what you’re going to see in the

keynote is the it’s coming full circle

we go from natural intelligence to

inspiring artificial intelligence to

developing the tools necessary to

analyze that to then applying those to

understand our own brains yet again full

circle

and that’s what you’re going to see but

you’re going to see live software demo

in 3D

right I hope

let’s pack that freaking theater you

know yeah

yeah I know it’s going to be I I mean

what you have described right now blew

my mind because it’s a lot of no well we

better hook you up to the software

though you know

I mean humans are really like to

recognize patterns but we are not so

good it’s a visualization so how do we

visualize even the trivial stuff in a in

a 3D space it’s completely complicated

yes now we’re talking about even more

Dimensions yeah so 12 24 512 Dimensions

yeah so we have techniques and so

at the end of the day the the visual

renderings are still in 3D it’s just

that it’s how you organize the data and

how you associate data together with

other bits of data and how you

how you give controls interactive

controls to the user so they can quickly

and easily not only switch between

different dimensions but also understand

the context exactly that’s that’s the

beauty of this tool because yeah you

know we’ve got some 3D surface

renderings we’ve got some 3D scatter

plot stuff and and we’ve got some 3D

trajectories but it’s not like it’s not

like that’s new other people have done

things like that in the past it’s the

new part and this is the thing that

really says Java is that uh

you’ve got to manage all that data and

all that information behind the scenes

and you’ve got it it is an effect it

isn’t useful for a couple dozen data

points or a couple hundred data points

it’s got to be like hundreds of

thousands of data points just to even

get started and it’s not just hundreds

of thousands of data points in one

direction it’s also hundreds if not

thousands of of it yeah you’ve got a

very long and wide problem and you’re

trying to then

roll it together and then display it in

3D but then also have it so it makes

sense like that that’s it’s got to be

fast it’s got to be straightforward

you put to you put Java effects together

with John it has to be beautiful well it

does I mean yeah nope

if you’re going to explore a problem

space

visually

and you got to stare at that it’s got to

be it’s gonna be beautiful because it

because it is the problem yeah the

pixels become the problem so it’s got to

look good it’s a representation of the

problem and we need to

and we need find this moment where we in

a way identified so yes exactly because

by by like that happens a lot with

the pattern first emerging and then what

the stress about the pattern and then

you get like the difference and and our

brains those those dots so fast that

sometimes we don’t we don’t even realize

our thinking process yes and actually

you know um another problem on top of

that is our our human nature to

interpret information that we see a

certain way because we’ve evolved over a

million years or whatever to see things

a certain way but the evolution that we

have

and I’m not an evolutionary biologist

I’m just this is just total Layman’s

understanding but we evolved to survive

we evolve to not get eaten right exactly

but that’s not these problems are

completely different from that right and

so the way that the data is presented it

matters because it has to it has to tap

in to the things that our human brain

have evolved to recognize but it also

has to be correct it has to be precise

and so that’s one of the the challenges

that I had with this tool another thing

is why I understand by your description

is you also have to

uh condensate consolidate a lot of data

and try to give it a different meaning

but in the same space that it’s been

going to really represent yes and allow

and allow the people uh manipulating the

different tools have this access and

Discerning how actually all this

different

Partners or different consolidations or

different matches or groups

yes yes absolutely uh I love it there’s

there’s a several different 3D uh

projections that you can see that

they’re like sub tools so we have we

have a 3D surface where it takes all the

data points that you have in time and

also all the the the width of the

hyper-dimensional vector you know the

width and then instead of treating it

like one big incomprehensible blob it

unrolls it and then lays it out as like

it looks like a landscape and then you

can you can enable these controls where

you can like you can hover over it with

the mouse and it it tells you precisely

where you are and what the numbers are

what they what the the minimums and the

maximums and the rows and the columns

are and so it

these are very generalized tools that

you might think oh you know yeah I’ve

seen something like that like I know

Matlab you can do that in in this map

plot live in python or whatever but okay

now do that for thousands and thousands

of points and do that in 3D and do it so

it it’s useful except and it’s like

suddenly a lot of these other tools they

just don’t get the job done you need a

Custom Tool and it’s got to be fast it’s

got to be

it’s gonna be pleasant to work with it

it’s got to look good and it’s also got

a it’s got to be inspiring and I I’m

saying that I’m not trying to be

pretentious it’s just if if if if

somebody doesn’t want to use your tool

they’re not going you know that you

already lost

all right and I don’t want to name any

names of tools that maybe we have to use

but we don’t like but they’re out there

you I have my opinions but yeah

in this way proud of what you’re

creating because what you are doing

right now it’s not trivial it’s not easy

we have talked a lot about user

experience user interfaces how do we

make like the basic rules and sometimes

in some projects we still see like oh

you’re adding where’s the delete button

do we need a little button and you’re

like

that happened to me just just that

happened to me last week

uh I had a guy there’s a couple guys

that are doing some Advanced um research

using some of the open AI large language

models and we have some internal servers

I’m actually going to talk about it uh

um that like cross-reference images and

text and in documents and stuff and you

do you can do like a Google like search

except that does all the

cross-referencing using AI on the back

end and they’re trying to visualize the

results in in this software which is

called Trinity you know Neo Trinity yeah

I know right yeah yeah yeah yeah

I hope I don’t get sued by whoever

Warner Brothers but

um so um

one of the things it does is it the the

software it tracks all the labels that

are in the data so it can tell you like

oh of all the data we’ve we’ve

identified six or seven different types

of labeled data and here here’s how

we’re going to color your your points

based on those labels right and you can

change the colors and stuff and what

they were doing was they were like oh

let’s try this data set now let’s try

this one they’re drag and drop and drag

and drop and it’s like bang and then

they clear and they do it again except

I never cleared the labels and so after

15 minutes of them just trying different

chunks of data and they they had like

like a hundred different labels in this

huge list and they’re trying to find

which one is in in the color they

weren’t sure which colors were what and

they’re like okay how do I clear the

labels and I’m like

uh you close the application and restart

it uh

I never thought of it there’s so many

little things in a user experience like

this and it’s in

the problem is um

you if you if you try to throw do

everything in including the kitchen sink

if you try to do it all up front you’re

going to fail yes yes because because

it’s too much and it’ll be maddening

and also frankly what are the odds that

a software engineer

who has a good idea

is going to get it right

such that all the people that might use

this tool are like yeah that was good

we I don’t know what exactly what they

want because I’m not them

so it’s better to do these things a

little bit at a time but then the caveat

the problem is exactly what you said

that there’s no delete button I I

literally didn’t have a delete button in

terms of labels

so I just added that last week

so that I’m able to actually give this

presentation there’s a couple things

there’s actually

I

finished some work on something last

week that uh uh that it’s something

called umap uniform manifold

approximation

P I don’t know what the P stands for

it’s a process for Dimension reduction

and it’s used very a lot lately in

machine learning to to go from like

hundreds of Dimensions down to maybe two

or three or four or five right it shows

clusters okay very useful okay yeah

I got an open source implementation

poured it over and working inside

Trinity and I hooked up a Java X guise

and control all the variables to it and

all that and you hit go and it bang for

thousands of points it just bangs it out

in a couple seconds right which is a lot

faster than the python version

um and then like and then boom here is

your new umap projected version of this

data and and we needed it for all this

AI stuff right

standard operating procedure since 2018

to use something like this in AI stuff

and that’s and this is like I finished

this on like Tuesday of last week right

one week out and I kind of sat back I’m

like

I wonder what would happen if I ran this

on the brink computer interface data

what happened you’re gonna find out oh

yeah

it looks

it’s cool

okay I’m going to be but it doesn’t look

like the the way it re clusters the data

and organizes it looks different than

how you you would normally have it with

like labeled AI data okay because trust

me and when I go back and I show the

Neuroscience guy and I’m like can you

use this and he’s looking at it and he’s

looking at it and his response was

I don’t know nobody’s ever done this

before and I’m like

[Music]

so so basically we’re in a situation

where like suddenly now he’s looking he

and his in his type have a new brand new

tool a brand new way to look at the data

that they are traditionally using and

working with that they’ve never seen

before and none of that would have been

possible if we hadn’t taken the time to

build up a rich framework a rich

software application with a with a very

rich easy to use visual interface

exactly that this is why you use these

powerful software development uh

Technologies like Java this is why you

it’s why visualization is important this

is why job of X is so important this is

why desktop applications matter

because it’s it’s not that you couldn’t

do bits and pieces of this unlike let’s

say webgl and like on like a web app you

could but like when you start to put the

sheer amount of data and all the things

that are firing all at once and managing

all that stuff all the uh

it’s not going to get the job done I’m

sure somebody out there is gonna be like

oh yes okay well you just don’t know

what you’re doing you’re like okay fine

but how much time do I need to spend to

to take some sort of technology that

isn’t good enough that it didn’t it

isn’t meant for something of this scale

and then try to make it work

I’m done I’m already done I have the

tool it’s done here and we’re looking at

open source in it in a couple weeks yeah

oh my God that’s going to be quite

announcement well uh it’s we’re still

going through that process so but I I’ve

been told that inherently it’s not going

to be a problem but there’s just the

paperwork I tried to get it done I

couldn’t it just takes a long time I

mean

it’s amazing because it’s no honestly

let me recap from the open like please

no software developer perspective no

the stuff that it’s way beyond my

imagination no me too I think

we’ll go to the keynote and then it’ll

it’ll be obvious after that you’ll be

you’ll be you’ll have a master’s degree

in whatever you know whatever I

tell them that I don’t know what I’m

talking about like

I just I write for Loops I write four

loops and I move pixels around that’s

what I’m good at

[Music]

uh Frameworks so that we can build on

top of them and they are fast resilient

reliable and and easy easy to extend

sustainable and sustainable yeah yeah so

I I like that of our

programming language and Java has been

has proven to be exactly that I’m very

mature it really has very slowly

perform

ance absolutely I

I’m sure you know you could do something

like in C sharp right you could probably

do something in like modern C plus plus

I’m not saying he could yeah uh you

could use kotlin you know but if it’s

just

it’s none of those languages are really

going to bring anything extra to the

table

except maybe performance with with C

plus so maybe if you do it right but

then

[Music]

then the ecosystem like the build tools

like like the security either

reproducibility yeah in that we are

super

man that’s why I’m still

I’m looking

for programming languages but my heart

is still on Java yeah me too because we

I’m

yeah I agree you’re preaching to the

choir yeah yeah yeah

I I got from here come from this amazing

conversation is like when you have the

tools

and you have a certain set of tools now

you can even ask new questions in the

domain because usually you you have like

let’s look at the stars and you you have

this idea and you’re building the tools

to look at the stuff yeah but suddenly

you have this amazing technology

and now it’s like what can we look at

yeah with this yeah you know it’s the

same reasons why people will celebrate

Dynamic scripting languages like python

Etc is that well I don’t want to worry

about all the all the machinations of of

the of the tech stack I just want to I

just want to get the job done

that’s great and I agree with that for

small jobs this is not a small job

because you’ve got to build up so much

capability

up front before you can even start to do

it real at work and Analysis

and and it and you I think it’s

important not all problems require that

that big upfront investment or at least

they don’t need to have that if you if

you can get an answer for your problem

in a more quicker fashion then you

should do it but I think we’re in such a

rush I think it’s a problem that as a

community as a society the software

development society and of course the

business that goes around it we’re in

such a rush to to make things so much

easier than they were before easier and

to do things so much faster and cheaper

that we we’re consistently hitting up

against brick walls of well we didn’t

invest early enough and enough effort

time and effort to build up a critical

mass of capability and so now it’s going

to cost triple that to go double back

and do it right so we’re just not going

to do it they’re basically they’re

selling out next year for next quarter

they want something next quarter so

they’re they’re just yeah and that’s

wrong but it’s also not my call it’s not

our call it’s their call and it should

be their call but what we can do and

what I’m trying to do here and in the

keynote level tomorrow morning

footsteps show them the value

prove that like put it that’s like bang

this took effort this took Tech these

Technologies made it so we could do it

and look what we’ve got now

and now you’re now if this if you wanted

to do this too you’re screwed because

you’re so far behind now that you’re not

going to catch up for a while now so

like meaning because they didn’t invest

up front there is value to research and

development there’s value to to

engineering that that pays dividends

later

and we need to show people the value but

I think I think culturally

especially in the west I can’t speak for

the east of the of of of of the world

um there’s there’s a disincentive

financially to doing that and I and I

that’s very short-sighted and I and I

think a lot of people kind of cave into

that pressure and they should they

should push back

parts of the world we measure even

Enterprises projects in months not even

years I know in other cultures it’s they

started thinking about like return of

investment we start at seven years and

and that pressure of course forces

software Engineers like us to make

choices that kind of uh Force us to do

things really quick and kind of on the

cheap it’s like well I’m not going to

fix this thing I’m just going to duct

tape it right it’s like okay yeah it

works today but is it going to work

tomorrow it’s not sustainable it’s not

sustainable but it doesn’t matter

because to the people making the

financial decisions that they don’t care

about sustainability we need as a people

the the push to be sustainable for the

environment needs to also apply to

technology yeah yeah because at the end

of the day if everything is as technical

as as we say it is it is it is very

quickly becoming the environment it’s

affecting the environment directly so a

little bit a little bit preachy here a

little bit philosophical but I I think

it’s important to say it here at the end

of this I mean we have a responsibility

we have a contribution to make a

software Developers yes

you’re taking a decision

yep you’re right yeah well thank you

very much and I’m so happy that you were

here and it’s a pleasure this amazing

session because it was almost a session

oh my God yes how long do we go 30

minutes wow

thank you very much no problem