Implementing C++ Semantics in Python @NDC TechTown 2021

0:05
okay hey good morning everyone my name is tamir and today we’re going to be talking about implementing c plus plus
0:11
semantics in python now before we begin and you probably saw this picture on the
0:17
website this was my breakout with hairstyle which was a lot shorter now it’s getting a bit longer but
0:22
it’s still me in the same person now and before we start i have a few
0:28
questions for you and who here uses c plus plus or raise your hand
0:33
okay and the primary language okay and who is this python
0:39
and the primary language okay now if you have any questions for
0:45
me there will be pauses during the talk for that and there are slide numbers on every slide so you can always refer to
0:50
those so the first question we have to ask is why on earth will they want to take c
0:56
plus plus semantics and get them in title i mean c plus fast is low level it’s mostly expert oriented and it’s
1:04
slowly becoming more and more pythonic as new features make it in python on the other hand is nice it’s high level it’s
1:09
beginner friendly and it has a lot less food guns i mean it happens that you write code and plan and go like oh i did not expect it to do
1:16
that at all and if you’ve been to the quiz yesterday i mean that’s not exactly a fun experience why would they want to
1:22
take that and bring that into python well at least for me the answer is resource
1:27
management in c plus plus all resources are treated equally eh there’s no special handling
1:34
by the system you just have to handle everything yourself in python on the other hand you have garbage collection so if you allocate
1:40
any memory it gets freed automatically and which is very nice but any other type of resource beta file socket db
1:47
connection whatever you do you need to handle it manually so it’s not like in c well you actually have to
1:54
write every call to close the results
1:59
just like
2:07
python does give us some facilities to do that namely context managers and context managers are fairly
2:12
straightforward whenever we use them with the with statement and then when we enter the within it creates an indented
2:19
block when we enter the block the enter method for that context manager is called and when we leave the block bit
2:24
by return statement just by getting the end of the block or we throw an exception
2:29
and we call the exit method for the context manager
2:35
if we want to implement our own context manager we just need to implement the internet methods and we have our own
2:40
context manager and we can do anything we want in those methods
2:46
so let’s talk about some real code and this has been redacted from some code i have running in production you know it’s
2:51
not the best code in the world but um you have code in production you have to live with that
2:56
and you pay for your mistakes anyway the code is fairly simple we have a zip file containing hundreds of files
3:03
and you want to be able to read them whenever we need to so initially the object is very simple we
3:08
open the zip file with the context manager in order to be sure to close it when you’re done we read all the files
3:14
from the zip file we unzip them and install them in the dictionary then we’re going to read in the actual contents just call the read method and
3:21
access the the data in the zip in the dictionary sorry
3:26
okay usage is very simple we create a new archive reader and then we call read get
3:32
the value print it and everything works as expected
3:38
now as time goes by things change in my case the archives got larger from hundreds of
3:45
files in the zip it go to thousands and tens of thousands and hundreds of thousands of files and that meant the
3:52
file was a lot larger and we could no longer just open everything and hold it in memory it was way too big to be
3:57
stored in memory in additionally the time it took to open the actual zip file just to pass the metadata go too long to
4:04
open the zip file whenever we want to read one single file so we had to do something we had to open a zip file
4:10
store it as a member variable in our class and only then read the files when we want to
4:16
this created a bit of a change in our archive reader now in the constructor all we have to do is open the zip file
4:22
we don’t need to read from the zip file yet in the read method we take the zip file we open a specific file within it we
4:29
unzip that file and return the data no need for a dictionary but to get that we also have to make our
4:35
own archive reader a context measure so that we can open the file and close the file whenever we need to do that
4:41
this means that we need to add an enter method and an exit method all fine so far
4:46
however this changes the user the usage of our class like we saw before with the zip file we use the context manager with
4:53
the with statement now we need to do that with our archive reader so if anyone’s used that archive we need to
4:59
change that code if the archive reader was used as a member variable annual we need to change that code as well and
5:05
that keeps propagating higher and higher and higher up the call stack and can pollute a lot of code
5:10
additionally if we cannot change any part of that code we would not be able to make the change
5:16
and use the new archive manager
5:25
well c plus plus has a solution for us namely the structures and in c plus the
5:33
factors of the same solution to the resource management problem and they have three main properties that we are
5:38
interested in first they are automatic then they are composable and they are implicit
5:43
let’s go over that first automatic whenever we have a distractor and we reach the end of a block this cycle is
5:49
called for the object that goes out and goes out of scope doesn’t matter if it’s because we reached the end of the block
5:54
and whether we flew an exception or we just returned and the cycle would be called just like with the context
6:00
managers then we have seamless composition if we have a class and it has no members everything
6:08
everything is okay our destructor would get called then if we add new member to that class and when the remember sorry
6:14
when the class gets destroyed the the member variable this structure would be called as well that just happened
6:20
seamlessly we don’t need to do anything to make that happen and additionally it’s completely
6:25
implicit here we have an example one object has this tactile the other one does not have a user-defined destructor
6:31
and the code is exactly the same don’t do anything special to use that um so if we just add a new destructor to
6:39
an object we don’t need to propagate a change higher up the call stack we don’t need to change anything it just works
6:44
which is wonderful so our goal is basically to take this
6:50
code which is our current archive reader which is 11 lines of code four of them are strictly resource management so
6:56
boilerplate and we create this code which is seven lines and no resource management at all
7:03
everything is automatic implicit and composable additionally we change the we change the
7:09
usage from this weird with statement usage to just you know regular variable usage
7:15
which is a lot nicer and again does not propagate the change higher up the stack we just added structure and we’re happy
7:20
to go before i continue and just a word of warn you are gonna do a lot of dirty
7:26
hacks here please do not try that at work if someone saw that in a code review they would be appalled and
7:32
probably call you to a conversation say hey what are we doing here that’s that’s not okay but we are not doing work here this is
7:38
for fun so on our journey towards a python that’s more like c plus plus we have our
7:44
greater class to join us it’s a very simple class just to help us help us visualize the constructor and destructor
7:51
calls when we construct the class it says hello and when we call the destructor it says goodbye it’s a
7:56
context manager just just like we did before and when we run it we simply get hello and then we have a
8:03
greater printed here and we say goodbye fairly simple now let’s make sure that everything is
8:09
automatic as i mentioned before in context managers are already automatic volume
8:15
explicit but everything happens automatically if we use the with statement we are sure that when we leave
8:20
the scope when we leave the block the destructor would be called now things get a bit messy if we want to
8:26
add another griddle and another greater because we keep adding nesting more and more nesting and it’s quite common for
8:32
us to have more than one object in our function like 10 objects is not that
8:37
well and as we keep adding that can get a bit messy
8:43
so what do we do instead of stacking everything in with nesting we can move to a propel stack
8:49
and so we create a new object called the destructor scope this cycle scope basically holds a stack with the
8:54
distractors for all the objects that we create and it’s fairly straightforward we use at least any other stack because it has
9:01
pop pen you can use the stack and like vector is the default container for everything in c plus plus in python we
9:08
use list and we have an enter method which just returns the same object that’s very
9:13
common with context managers we usually don’t need to do anything in the enter method and you have an exit method that goes
9:19
over the stack in reverse order and distracts all the objects so we actually call it the cycles in the right order
9:25
and we have a push method to add to the stack when our code changes to this we create
9:30
a scope and then whenever we create a glitter we push that into the scope and
9:36
when we leave everything gets called in the right order now this is nice but that’s
9:42
very very explicit i mean i don’t want to write all that code for every single object i add for every single function
9:49
in my code so next step is to try to make things more implicit any questions so far
9:56
okay
10:04
basically we want to remove all if we cannot remove hide all the boilerplate in our code we want to change from the
10:10
factor that we explicitly create the destructor scope we give it a name create the griddle we push them
10:15
explicitly onto the stack and then it gets selected to just
10:21
the simpler code inside we create a glitter and that’s it it just works
10:26
so whenever we want to hide something we usually add another layer of interaction in our case
10:32
this is what we’re going to do first things first um if we don’t want to push this in greater explicitly and you don’t
10:39
want to forget to push the glitter then we just make sure that when we initialize the glitter it needs to get the structural scope if it does not get
10:46
that we cannot create the object so move in the pushing into the constructor
10:52
and change the code a bit so now it’s a bit more it’s a bit less code but
10:57
we still need to name the suctile scope and i don’t really like that i want to hide that name
11:03
so let’s say we have the scope we want to hide it we want some other
11:09
variable to have a functional class to have access to it well i guess the best solution is a global variable
11:16
though they’re always fun but since um functions are being called on a stack
11:23
and keep adding new scopes we’ll have to make that global variable a stack as well so we’ll have a stack of
11:29
these tactile scopes like we created before and whenever we enter a new function we just push that onto the
11:34
stack so we change over the cycle scope a bit to make sure that when we create a new
11:39
scope we push it onto the global stack and when we destroy the scope we pop it off the stack
11:45
fairly straightforward we add a new global function to push something onto that stack
11:52
and then you can use it in the in the gripper so instead of getting the desktop stack explicitly we just use the
11:59
global stack and we’re good to go now we don’t pass any more variables into the stack
12:05
and our main basically looks like that we create a new scope because we have to do that we create the griddle they get
12:12
pushed automatically onto the stack and when we leave main they get distracted automatically
12:17
and but we can do better we can do a lot better first um get this tactile scope i
12:23
don’t want to create it in every function i call it’s quite annoying so first thing you can do since it doesn’t
12:28
have any code that’s related to the actual main function it only uses global variables and let’s move it outside of
12:34
the main function this works exactly the same and now we just need to do it around the recall and
12:39
not inside every function it’s still not the best and so let’s create a
12:44
new way to call functions we create a call function it takes a function and it’s arguments and the style styles are
12:51
kw args is python’s way of doing perfect folding so we take a function we take the
12:57
arguments and then we perfectly follow the arguments into the function and after creating a destructor scope so
13:04
here we call main and it works just like before but it’s a bit simpler but still i i don’t really like calling call
13:11
whenever i want to call a function like i wouldn’t use invoke every time you call a function in c plus plus
13:17
so we go another step of interaction okay the cpp function function and it
13:24
takes a regular python function and returns a c plus plus function so inside
13:30
we have a closure we take the function that we passed we capture it in the closure we added the structural scope
13:35
and returned the closure then we have a new function called scoped main which does the same thing as main but it’s
13:40
wrapped in instructor scope and then since python is a dynamic
13:46
language after all and we can rebind names to whatever we want we can rewrite the name main to the new scoped
13:53
version of the function and now if we call main it creates the scope and calls the original main we defined
14:00
and this is a fairly common operation right on taking one function wrapping it in something else and
14:06
assigning it to the original name so if i don’t edit the new specific syntax for that called
14:11
decorator syntax and basically instead of calling the function directly we just use at and the
14:17
name of the function above a function definition and it does the exact same work takes the function passes it into
14:22
the function takes the return value and assigns it to that name okay so now we can write the main function this way and it works just like
14:29
we did initially it created this suction scope make sure that thing gets pushed and returns the value properly
14:36
i think this is already a lot better than where we started it’s declarative which is nice it doesn’t have just weird context managers
14:43
lying around it’s fairly clean the contents of the main function look like a perfectly regular function
14:50
but it’s still explicit and again i have to write it every single time for every single function and we
14:57
can probably we probably want to avoid i mean if i forget doing that and you just try the main function without it we
15:03
would not have this cycle scope and as we write more and more functions that could become annoying especially since
15:08
if you forget it for one function you just bind to the function above when things look really weird
15:14
any questions so far
15:37
okay now we’re going to start with import hacking this is where things get really messy and fun
15:43
so basically what happens in a file if you want to use our new library with the cpp function um in our own code first
15:50
thing we do is form cpp which is how we’ll call our c plus plus library that we use and we import the cpp function
15:56
decorator then we import the glitter then we create the main function and use the cpp
16:02
function decorator fairly straightforward now what if i want to do something else what
16:07
if i just want to import magic form c plus plus and have everything else just happen
16:13
um well let’s start small let’s say that instead of only importing magic i also call the magic function and i want that
16:20
point to do whatever is necessary in the file and to get the same results as we’ve gotten before
16:26
so what does the magic function actually have to do needs to do two things one is find out which is the calling module
16:32
where was the function called and so it needs to modify that model and decorate all the functions within it
16:39
let’s see how to do that in the magic function we use the
16:44
inspect module to get the call stack then from the call stack we get whichever module called us
16:49
once we have that we can return the module it’s very nice that we have that functionality in python i wish someday
16:54
we’ll have call stock inspection in c plus but that’s probably a long way off
17:00
then we need to decorate all the modules functions so basically we go over the func at the model and iterate over all
17:05
the modules members the first thing we check is is what we’re looking at a function if it’s not a function we don’t
17:10
need to do anything then we make sure the function was defined in that model so only if we imported like something
17:16
from the standard library we don’t need to decorate it we don’t want to change the functionality of the standard library that would cause very weird
17:23
behavior and then we set that attribute back into the module we just use setup which
17:29
allows us to take an object a name and a new value when we fold it attribute and we call this if we function on the
17:35
function and to decorate it like we did before and with that we can write the code as
17:41
follows and we just import magic define all of our code call magic to make the
17:46
magic happen and then call main everything just works
17:52
but it’s quite annoying you have to explicitly use magic magic should be magical and visible and fun
17:59
so what what do we need to do to actually
18:05
oh this was confusing i have a duplicate slide sorry so what do we actually need to do and to
18:11
make the magic call disappear well for them we need to talk about the actual import mechanism in python
18:18
this is a basic diagram of what happens when you write from cpp import magic this is the import expression and you
18:24
want in the import statement and you want to import magic from the c plus plus module the first thing we do is
18:30
check whether the c plus plus module is in the global module cache if it is in the cache we just move ahead if it’s not
18:36
we need to go to three steps first we need to create a new module object then we need to add that model to the
18:42
cache and then we need to execute that module and generally python adds the model to the cache to prevent cyclic
18:48
imports from just re-importing it again and again and again which is quite unpleasant once we’ve executed the model and check
18:56
whether magic is in the model if it is in the model we just find the magic to the name magic in the importing module
19:02
if it is not there we call the getatter attribute and the category method of that module if that exists and pass the
19:09
name that we’re looking for which is magic since magic was a function call we probably want to use a function call as
19:15
well um so that’s what we’re gonna go for
19:21
we change the name of the function magic to underscore magic to make sure that we don’t accidentally just import it
19:26
directly and then we create a get outer method in there we check whether the name is magic
19:31
or not if it’s not magic no magic happens if it is magic and we call the
19:36
magic function and now everything should work we import it the magic function is called
19:42
everything gets decorated and we’re done except there’s a bit of an issue
19:49
if you see in the output here we got hello one hello too but no goodbyes
19:54
um well if you look at what we’re calling the magic function we call it
19:59
the first line of the module this is before we define the main function so when we import the model we check for
20:06
the calling module we get our main model and then we check for members in that model it’s empty so we decorate all the
20:14
nothing and nothing happens so we need to do something else
20:20
one thing we can do is move the import down the line if we do it the same place we called
20:25
magic before it works but i don’t know about you that does not look magical to me
20:30
that’s iffy so we need to do something else well
20:36
python allows us to do parallel imports which means we can just instead of using the import statement we can write our
20:43
own code to import new code and again write the function it goes by the same way and we pass in the name of
20:49
the model we want to import but the path path to that file and so that would find
20:54
it and then we create a module object we add it to the cache and we execute it first forward
21:02
now we’re going to do something a bit more fun we get the calling model then we take the name of the path and
21:09
the path of that model and import it again since we’re executing the module while
21:15
we import it we will get the model after it’s fully executed which means all the methods in it have been defined
21:21
now we can decorate everything and we should be okay
21:26
so the magic happens and we have a recursion error because we
21:32
imported the module and then the module imported magic again so we imported the module again so we the module imported
21:39
us again kept going going going until python fill up the stack a python unlike
21:44
in some languages it does allow for cyclic imports it’s something useful you just need to make sure that they stop at
21:50
some point yeah otherwise you get a recursion l so we need to do something about that
22:08
in clc plus plus we’d use evil pogba ones or include guards here and i guess
22:13
we’ll use an include guard as well when we import the model using our parallel import function
22:19
we can add a flag into that module then when we execute the magic we check is the flag
22:25
does the flag exist or not if the flag exists we don’t need to do any magic because we’re doing it
22:30
externally and we can just continue and this prevents the occasional
22:35
so now everything should work right well no
22:41
what happens now is we import the magic we create a new model we decorate everything great but
22:47
during the import we execute the module when we execute the model the main function is not yet decorated which
22:54
means that when the gritters try to push themselves onto the global stack there’s
22:59
nothing there and we crash so we need to change the way we call the
23:05
main function this is fun because it allows us to do something a bit more like c plus plus in python whenever you double click
23:11
about a python file or executed for the command line the name of that model is going to be underscoring main underscore
23:18
underscore and this means we can check for that to find whether the module is currently importing magic is the main model or a
23:25
different model if it is the main model we just check whether there is a main function inside that model and if there
23:31
is we would execute that function once we finish executing it we just exit so essentially we check for the main
23:37
function we do this we run it just like c plus plus and this means that you can also remove
23:43
the main call from our function we just create the main function and everything works
23:49
um any questions okay
23:55
so let’s look at the greater again um so far we’ve created this one class but
24:00
we’re going to write a lot of classes not just the graded class so if we have here like six lines of boilerplate we
24:07
might want to be able to remove them we create the init which currently just pushes us onto the stack which is quite
24:13
pointless and we have the enter function which does basically nothing and we have
24:18
the exit which called which is the destructo you want to reduce some of that boilerplate now since this is a
24:24
class and we’re an object-oriented language well the obvious thing to do is just create a base class to do the work
24:30
for us so if we have a base class that does something we can create a constructor
24:36
here named as the same same as the function and it operates like a regular constructor and we have the structure
24:43
sorry python does not allow me to use the tilde in a variable name so we have underscore instead still in my opinion
24:50
better than under swan also exit and we have the destructo and that’s basically the entire reader class and removed all
24:57
the boilerplate as for the cpp class class we we have a constructor with suctal and
25:04
the useless end of any method in the constructor we push ourselves under the destructor scope and then we
25:10
check either a constructor for the class if there is one we call it then in the destructor we check either a defined
25:16
destructor if we have this factor we call it this factor and that works
25:24
additionally in we might have methods in our class and we want them to be decorated the same way that we did with
25:30
the main function which means that just like we went over the module object and and over all the
25:35
members to look for functions and decorate them we do the same with the class itself go over all the members of
25:41
the class and check they do not begin with a double underscore because that’s special methods and we don’t want to
25:46
handle those don’t want decorate the dandelion also exit in it or enter
25:52
then we make sure that that thing is indeed the function not the variable and then we decorate it just like we did
25:58
before and let’s see where we’re standing now we have a greater class
26:04
it inherits from cvp class and basically works but that that cvp class
26:11
is is still a bit explicit i mean you have to write it all the time and i want it to look just like python code to be
26:17
completely implicit i mean in c plus plus when you have a class we don’t need to say oh and and act like a regular
26:23
class we just write class and it works so what actually happens when we use
26:30
inheritance here and one very simplistic way of looking at it which is good for our case is
26:36
basically we take all the methods from the base class and copy them over into the subclass
26:41
now if we’re just copying methods around and we’re using python which is a dynamic opening language it allows us to
26:48
change objects at runtime we can just assign the methods directly
26:53
instead of using subclassing so instead of having a cpp class class
26:59
we can have a cbp class function that takes another takes a class and converts it into a cpp class
27:06
just like we created before we define all the init enter and exit methods we decorate all the current object methods
27:13
and then we just assign the new init accidental methods to our class and
27:18
return the new class this works then classes can be used with decorative
27:24
syntax as well so just use the cpp class function as a decorator and convert our
27:30
griddle to a cpp class now one last step we have to do here
27:35
in since before we just check oh this is an object is it a cvp class well you can
27:41
check for a using easy instance the checks for inheritance in python and you can see that the object was indeed an
27:48
instance of a cpp class now that we are not using subclassing we cannot do that so we had a small flag in the object that says yes this is a cbp
27:55
class not a regular python object
28:06
with that we can go and back to the magic function we wrote before and at the similar method function to decorate
28:12
all the fun at the module classes so again we go over all the members of the module
28:17
we check whether they are classes and unlike the functions we did before we make sure they were defined in the same
28:23
module and then we decorate them just like we did with the functions so now our greater looks like this and
28:30
no hint of context management or resource management anywhere our main is fairly straightforward
28:36
and everything just works any questions
28:42
okay so let’s go over what we did so far in first everything is automatic and
28:48
selection is automatic whenever we leave a scope we destroy everything that needs to be stored destructed in that scope
28:54
and second it’s implicit and we just import magic forms c plus plus and everything else just happens we don’t
29:00
need to write any spatial code no don’t need to do anything specific to make context management a resource management
29:06
work and additionally the main function is just called which is a lot more fun
29:11
our next stop is composition
29:17
so let’s look back to where we started we had 11 lines of code four of them were
29:22
resource management now we have nine lines of code and two of them are resource management i think that’s already better
29:29
but we have an issue and the cycle is currently called twice we create the zip file assigned to a
29:35
local variable to a member variable and leave the constructor as soon as we do that the zip file gets distracted
29:41
because we left the scope then when eventually we go to the destructo
29:47
of the archive it’s called again and we actually destroy it twice
29:52
it would probably be okay because it’s python and we don’t have any dangling pointers or anything like that but we’re
29:58
still using uh already distracted object which would probably be closed and nothing would work
30:04
so we need to fix that simple way to fix that would be just to remove it from the current scope
30:10
yeah create the member we’ll create a variable assigned to a local member and then remove it from the desktop scope
30:15
and we know it would not get destructed automatically when we leave the function now how do we do that
30:21
whether we move function and remove method toward the stack or scope maybe call
30:26
stack which is the list dot remove and pass in the member and we have one issue here list dot remove is a quality based
30:33
and not identity based so if you have two different context measures with which equate two
30:38
and we may destroy the wrong one we remove the wrong one which is not a very pleasant thing to do
30:45
so we need to take something from the c plus plus playbook and create a customization object here we create an
30:51
identity comparator it basically takes an object and overrides the
30:56
eq method which is what python uses for equality checks and we use the ease statement is a check here
31:03
which is identity it checks identity and not equality then we can pass that to our remove
31:08
function and then instead of checking for equality it would automatically check for identity with all the objects
31:14
and things just work as we want them to do
31:19
now we can call remove and we’re good to go but this is still explicit and we are
31:25
working on making everything implicit and transparent
31:31
so um if you have to do something every time we’re assigned to a variable or get the value from a variable the obvious
31:37
thing to do is to use getters and settles so in get zip file and we just return the value from the right
31:43
attribute that’s pretty straightforward because we don’t need to do anything when we set the zip file over there
31:48
several steps first if there is already a value in that member variable and we need to distract that value because
31:55
otherwise we’ll have a dangling object and then once we do that we can assign
32:01
our new member and remove it from the destructor scope and again like we had before with
32:08
descriptors sorry with decorators a python has a special syntax following
32:15
ghettos and settles called descriptors a descriptor has to be a class member variable not an instance variable
32:22
it has to implement the gets and set methods and it may also implement the set name method which is used to let the
32:28
variable know which variable name it’s assigned to so specifically here
32:33
when we initialize cppml when the class itself gets created not an instance but the class type
32:40
set name would be called for the cpp member instance assigned to zip file variable with the name zip file so that
32:47
it knows oh i’m the zip file variable and not a different variable and to create the cpp member class we
32:54
implement the free method methods we have set name which takes the name given to it and stores a private name since
33:01
this is a class member variable not an instance member variable and we have one for the class if we have
33:07
multiple instances it would still be the same cpp member instance so in order to actually store values in
33:13
the class in the instance we give it a new name and assign directly to the
33:18
instance with that name then in get we access the instance with the name we created with the
33:25
underscore prefix and when we get the value and when we set the value we do just what we did
33:30
before but again using getter and setup with the private name we created
33:36
any questions okay and
33:42
and then everything just works we assign to the variable it gets removed from the scope it does not get automatically distracted
33:48
um but we still called the desktop zip file explicitly in our destructo and in
33:54
c plus plus we don’t need to do that so we probably shouldn’t have to do it in python evil um
34:00
so we add something else to the exit method we define before we go through all the members of them of the class
34:08
and we check whether we start with underscore if they do they’re the private members and we don’t want to touch those because then we distract
34:14
them twice and we check whether the value currently stored in the member variable is a cpp
34:19
class if it is not it does not have a distributor we will have nothing to call and if it is we call the exit method we
34:26
call it destructo and carry on so essentially we just automatically call all the destructors of all the
34:32
members once we finished calling our disciple and now the class looks like this
34:39
we have zip file it’s a c plus plus member and no more resource management code but
34:45
there’s still something i don’t like here and this is explicit and i mean i’m writing python code why should it say
34:51
cpp member that makes no sense uh so we can do better python has what’s called type annotations and patients are
34:58
a wonderful thing they do absolutely nothing they will just type in sorry
35:03
objects that can be attached to different things they can be attached to variables they can be attached to
35:09
functions classes return types function arguments everything can have an attribute annotation assigned to it at
35:16
runtime it does absolutely nothing it’s sometimes static type checking
35:21
but it is stored and it is accessible at runtime which is the important part
35:27
now um what we want to do now is instead of assigning the members directly we go to our cpp class method a function and
35:35
add a function to create all the members what you do is we go through them patients and for orientation we assume that this
35:42
is a new member variable and we create a new member a cbp member we create the cpp member object since
35:49
the class has already been created if set name would not be called automatically so we call it ourselves
35:55
with the name of the variable we got from the annotations and then we assign it to the class in
36:00
the same name and additionally we save all the member names and for later use
36:07
then in the exit method in our destructo instead of going for all the members of the class and checking whether they’re
36:14
variables or not we just go over the names and already know those are member variables because that’s what we defined
36:20
and so we distract them and now you basically done we have seven
36:27
lines of code and no lines of resource management and everything just works very
36:33
transparently now you might claim that the zip filenation is for resource management but then i would say no it’s
36:39
for the building you want to set the type so everyone would know what happens there and
36:44
since it’s my talk you have to agree so what did we do so far
36:50
um everything is automatic and the suckers are called when they are needed we don’t
36:56
need to do anything for that everything is composable if we add the new member then it just folks we don’t
37:01
need to add any special code for handling that and everything is implicit assuming you
37:08
write a specific type of code i mean you need to import the module and the magic form c plus plus
37:14
and you need to make sure that you write the constructor and destructo the right way and use annotations if you do that
37:22
everything just works with wonder for c plus plus the structural semantics and you’re basically good to go and we don’t
37:28
need to add any special code to our code there’s no change in the interfaces when we add the new distractor and it does
37:34
not pollute the interface as we go higher up yes that’s lying because we have to
37:39
write all of the code in this method and if you just use regular python code nothing would work
37:46
but assuming you write all your project this way if you add a new constructor and distract your code you don’t need to
37:51
make any foldable changes um any questions
37:58
okay before we continue to some bonus content i want to thank a few people a bracket
38:04
kin for helping me with this talk and going over my content again and again and again to make sure everything is
38:10
okay edition by levy from the cppcon conference is it called cpp sorry
38:15
conference that encouraged me to submit the talk and the title and the sister for helping with
38:21
the many many annoying questions i had before going here and now for extras and we’re going to
38:27
cover three things one in return values two this and three member access specifiers
38:34
and this is going to be fine with live coding
38:40
oh can you see
38:48
okay can you see the cuddle should they make it larger
38:56
battle oh should be louder okay great
39:02
so the first thing we want to do um for when we created a function and we created some variable inside the
39:08
function and then we returned the variable from the function and the variable got destroyed
39:14
because we left the scope we call it the selector and that’s not very useful for anything i mean a caller function to
39:20
create an object i want to get that object back in a working state otherwise what am i going to do in my code so
39:28
we can solve that fairly easily and this is the cpp function decoder we used before and it has a bit of extra code
39:34
but we’ll ignore it for now we call our function and we get the return value once we get the return
39:39
value and we know that it left the function so just like what we did before
39:45
so we’ve member variables we know we can remove it from the current scope however after we remove it we need to
39:51
assign to a new scope before we assign it into the class scope now we don’t have a class code so we need to assign
39:56
it to the parent scope just define the calling function do it using rebind to parent this cycle
40:02
and we check whether the function is a cpp class function and whether the object is a cpp class object if it
40:09
is we remove it from the current scope and push it into the parent scope then we can return values freely
40:16
passing arguments however is a completely different subject and we are not going to cover that
40:21
essentially a lifetime some lifetime semantics are a big and complicated subject that
40:28
we’re not going to delve into because even in c plus it’s complicated now the next thing we want to do
40:36
is this i mean i know that some of you are python programmer but some of you are c plus
40:42
plus programmers and coming from c plus plus to python or from basically any object-oriented language to python it’s
40:47
quite annoying you have to write self everywhere we have a function takes no variables it takes no arguments itself
40:53
you have a function takes some arguments self and then everything else it gets quite cumbersome to write it all
41:00
the time and we we just don’t want it but it would be a lot nicer if we could just remove that annoying thing and you
41:06
know just access this so we’re going to use the same tricks as we did before
41:13
and naturally a global variable because those are the best we call this so it’s defined
41:19
everywhere and we can just use it and
41:25
then we need to somehow know what object is that this object so we had another
41:31
scope just like we did before with the destructor scope we created this scope this scope is specific to a methods and
41:38
not functions so now we have a method decorator and a function decorator which works slightly differently
41:45
and you want to create this scope we take the self object and push it onto a global this stack
41:52
again global stacks are a wonderful wonderful thing and maybe not in production code
41:59
and then because in python we not have a copy assignment we don’t have assignment operators or anything like that when we
42:05
assign the new variable a new value to a variable we just rebind the name nothing else happened no call gets
42:12
called just a pure assignment and we bind the name so if instead of
42:17
uh using a proxy we just assigned to the this variable and nothing would work to
42:23
just be a new variable and nothing would actually happen instead we use this proxy
42:30
the this box is fairly straightforward it uses a special method in python get apple um allows us to hack yeah sorry to
42:40
hook into the member access in python if the name of an object of an attribute is not found will you
42:46
call the getaterma object just a sec
42:56
we call get utter and underscondal to get the attribute of the object
43:02
using our code in this case we would get to the top of the d stack ask for the name of the attribute using get
43:08
attribute and return the same full setting and then you can write code that uses this
43:14
everywhere and completely ignores self additionally
43:24
as you can see we we’ll no longer we capture the in self variable in our wrapper and we just don’t fold it into
43:30
the function so now we can just write the greater this way and
43:35
we have we just pass in the name no self use this and everything will just work perfectly
43:41
which is really fun
43:46
now one thing that in my experience um c plus plus programmers really hate
43:52
about python is access specifier i mean in cprs plus
43:58
we have public private protected and it actually does something titan well it’s a bit different
44:04
if you want to create a public method we just create a method wonderful everyone can access it and we’re good to
44:11
go if you want to create a private method
44:18
we just append and underscore prepend and underscore and then everyone knows
44:23
it’s a private method and they should never ever recall that which means that if they accidentally
44:30
need to access some inter something internally in your class they would definitely call that and you would never be able to change that code ever again
44:37
so python gives us another option which is using a double underscore
44:42
when we’re using a double underscore every access to that function from outside the class would have to use that
44:49
mangled name which is underscore so name of the class another underscore and then the method that would absolutely prevent
44:55
people from using it right i mean you’ve never hacked anything weird in your code to access some something internal even
45:02
not in python not in c plus plus not in any language that you’ve ever used so so we might be able to do better to
45:08
actually have excess specifiers
45:16
something like this maybe well name is private and the rest is public
45:22
so what are we going to do here this is a bit more complicated it’s going to include most of what we did so
45:29
far it also doubled the size of my library from around 180 lines to around 360.
45:38
because it’s a lot simpler than the rest of the code so first we need to create our x specifier to have public private
45:43
and protected as one does our default access specifier is of course private because we want to be on
45:49
the safe side and we have um some global access
45:54
variable to store the access now we want to create a function a class
46:01
the code goes as follows we start from the top and start defining things here we have a name and it
46:08
doesn’t have a value only an annotation so it goes into the annotations dict then we call public we’ll get to that in
46:14
a moment and we define some functions functions just go into the class and we have functions
46:19
additionally everything that we define here is a index is in a new scope that’s the class
46:25
definition scope which exists as long as we’re defining class we can run any code we want in here
46:31
and for example we’re calling a function inside the class definition it’s called when we define the class not when you
46:37
instantiate it so what do we actually do here
46:48
well obviously we set the access which goes as follows and whenever we
46:54
call set access we go through all the members that already exist in the class we go output we use the stack to get the
47:00
local variable stockhold and take all the variables from bell and the annotations we check which variables
47:06
are new in the and all the new ones get assigned in the current same project and
47:12
access level and we save that in new value in the access the dictionary that we defined
47:18
earlier and with the name of the current class so that we know those access pacifiers
47:23
belong to those variables inside this class so basically when we go for the little
47:29
and once we call public name would be set to private because it’s before the call to public
47:35
and nothing would happen to those they would stay floating around with no expensive files
47:44
then in cvp class when we create the class methods
47:49
we also get the member access for every class and pass it to the cpr method function
47:56
think class class and method function and class function or that gets a bit
48:01
confusing and so basically check the global access dict
48:06
for the current class if not none exists we just take a default one we should just have a private access specifier
48:13
and then for every member in the class we get for that member name the current specifier if none exist we default to
48:21
private so if we just create one function we didn’t call any access specifiers it would be
48:26
private and one switch and since each time we call access pacifier we change
48:32
the current specifier for the class to whatever specifier it was in this case public
48:37
so when we create the class those functions would be public
48:42
any questions so far okay also since this is live coding stop me
48:50
if i’m missing something or if you need any explanations
48:55
then in cpp method we need to check the access and we know that we will call we
49:01
need to know who called us and how so to know where we will call the form and we create a color scope
49:07
and just like the previous scopes it’s a stack it’s a global stack that solves the colors
49:13
it can either store a function caller or a class caller if it’s a class caller we add the class itself and if the function
49:20
caller we just add the function then we call check access we get the
49:26
current caller so that we know who called us and comp and check whether the current caller can access this specific
49:32
function and with the current access so basically go if the caller is none
49:39
which means we are up to a regular python function with no cpp functions in the middle just yeah yeah we can use
49:46
that because i want to be able to write tests and i want to be able to use my code for python code this is completely unaware
49:52
of all my shenanigans then if it’s public we can always access
49:57
if it’s protected we check whether the current class is an instance of the same class as before and this also works with
50:03
subclassing so we’re on the safe side and if it’s private we make sure those classes are of the same type
50:10
if none of those things apply just return false
50:17
and this is what happens for methods for variables
50:23
and we have our cpp member and we call check access on the get and the
50:28
set methods so we get basically the same behavior and
50:34
both for functions and for variables now we can go to the greater and we know
50:40
that this is public so we can run the code and everything works however
50:45
if i remove the public and call it again we get an xsl because we’re trying to
50:51
call a function that is private
50:57
and that’s basically it any questions
51:03
okay thank you [Applause]
51:12
and also if you want the code is online and the talk the slide is going to be online as well and thank you for coming
51:26
you