If You Create an Objects Thats a Pointer and Then Set It to New Again What Happens
Spotter Now This tutorial has a related video course created past the Real Python squad. Picket information technology together with the written tutorial to deepen your understanding: Pointers and Objects in Python
If yous've ever worked with lower level languages like C or C++, so you've probably heard of pointers. Pointers allow y'all to create bang-up efficiency in parts of your code. They also cause confusion for beginners and can atomic number 82 to diverse retentiveness management bugs, even for experts. Then where are they in Python, and how can you simulate pointers in Python?
Pointers are widely used in C and C++. Essentially, they are variables that hold the memory address of some other variable. For a refresher on pointers, yous might consider checking out this overview on C Pointers.
In this article, you'll gain a amend understanding of Python'southward object model and acquire why pointers in Python don't really exist. For the cases where you need to mimic pointer behavior, you'll larn means to simulate pointers in Python without the retentiveness-management nightmare.
In this commodity, you'll:
- Learn why pointers in Python don't exist
- Explore the difference betwixt C variables and Python names
- Simulate pointers in Python
- Experiment with real pointers using
ctypes
Why Doesn't Python Accept Pointers?
The truth is that I don't know. Could pointers in Python exist natively? Probably, simply pointers seem to go against the Zen of Python. Pointers encourage implicit changes rather than explicit. Frequently, they are complex instead of simple, especially for beginners. Even worse, they beg for ways to shoot yourself in the human foot, or exercise something really unsafe like read from a department of memory you were not supposed to.
Python tends to endeavour to abstract abroad implementation details like memory addresses from its users. Python often focuses on usability instead of speed. As a result, pointers in Python don't really make sense. Not to fear though, Python does, by default, give you some of the benefits of using pointers.
Understanding pointers in Python requires a brusk detour into Python's implementation details. Specifically, you'll demand to empathise:
- Immutable vs mutable objects
- Python variables/names
Hold onto your memory addresses, and let's get started.
Objects in Python
In Python, everything is an object. For proof, yous can open up a REPL and explore using isinstance():
>>>
>>> isinstance ( 1 , object ) True >>> isinstance ( listing (), object ) Truthful >>> isinstance ( True , object ) Truthful >>> def foo (): ... laissez passer ... >>> isinstance ( foo , object ) True This code shows you that everything in Python is indeed an object. Each object contains at least three pieces of information:
- Reference count
- Type
- Value
The reference count is for retentivity management. For an in-depth await at the internals of memory management in Python, you can read Memory Management in Python.
The type is used at the CPython layer to ensure type safety during runtime. Finally, there's the value, which is the bodily value associated with the object.
Non all objects are the same though. There is i other of import distinction yous'll need to understand: immutable vs mutable objects. Understanding the difference betwixt the types of objects actually helps analyze the get-go layer of the onion that is pointers in Python.
Immutable vs Mutable Objects
In Python, there are 2 types of objects:
- Immutable objects tin't exist changed.
- Mutable objects can exist changed.
Understanding this departure is the first key to navigating the landscape of pointers in Python. Here's a breakdown of common types and whether or non they are mutable or immutable:
| Type | Immutable? |
|---|---|
int | Yes |
float | Yes |
bool | Yeah |
complex | Yeah |
tuple | Yes |
frozenset | Yes |
str | Yes |
listing | No |
set | No |
dict | No |
Every bit you lot can see, lots of commonly used primitive types are immutable. You tin prove this yourself by writing some Python. Yous'll need a couple of tools from the Python standard library:
-
id()returns the object'due south retention address. -
isreturnsTruthfulif and only if two objects have the same memory address.
One time once again, you can use these in a REPL environment:
>>>
>>> x = 5 >>> id ( x ) 94529957049376 In the above code, you accept assigned the value v to x. If you tried to change this value with addition, then you'd get a new object:
>>>
>>> x += ane >>> 10 6 >>> id ( x ) 94529957049408 Even though the above code appears to modify the value of ten, yous're getting a new object as a response.
The str type is also immutable:
>>>
>>> s = "real_python" >>> id ( s ) 140637819584048 >>> southward += "_rocks" >>> south 'real_python_rocks' >>> id ( s ) 140637819609424 Over again, s ends up with a different retentivity addresses after the += performance.
Trying to directly mutate the string s results in an error:
>>>
>>> s [ 0 ] = "R" Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'str' object does not support item assignment The above code fails, and Python indicates that str doesn't back up this mutation, which is in line with the definition that the str type is immutable.
Contrast that with a mutable object, like list:
>>>
>>> my_list = [ 1 , 2 , iii ] >>> id ( my_list ) 140637819575368 >>> my_list . suspend ( four ) >>> my_list [ane, two, 3, 4] >>> id ( my_list ) 140637819575368 This code shows a major departure in the two types of objects. my_list has an id originally. Even after iv is appended to the list, my_list has the same id. This is because the listing type is mutable.
Another way to demonstrate that the list is mutable is with assignment:
>>>
>>> my_list [ 0 ] = 0 >>> my_list [0, 2, 3, 4] >>> id ( my_list ) 140637819575368 In this code, you mutate my_list and set its first element to 0. However, information technology maintains the same id even after this assignment. With mutable and immutable objects out of the way, the next pace on your journey to Python enlightenment is understanding Python's variable ecosystem.
Understanding Variables
Python variables are fundamentally different than variables in C or C++. In fact, Python doesn't even have variables. Python has names, not variables.
This might seem pedantic, and for the most role, information technology is. Most of the time, information technology'due south perfectly acceptable to recollect about Python names as variables, but understanding the departure is important. This is particularly true when y'all're navigating the tricky subject area of pointers in Python.
To help drive home the deviation, you can take a look at how variables work in C, what they stand for, and so contrast that with how names work in Python.
Variables in C
Let'south say yous had the following code that defines the variable ten:
This ane line of code has several, singled-out steps when executed:
- Classify plenty memory for an integer
- Assign the value
2337to that retentivity location - Indicate that
xpoints to that value
Shown in a simplified view of memory, it might look similar this:
Hither, you can see that the variable 10 has a fake memory location of 0x7f1 and the value 2337. If, subsequently in the program, you desire to change the value of x, y'all tin practice the following:
The above code assigns a new value (2338) to the variable x, thereby overwriting the previous value. This ways that the variable 10 is mutable. The updated retentivity layout shows the new value:
Notice that the location of ten didn't modify, just the value itself. This is a meaning point. It ways that x is the memory location, not simply a name for it.
Another way to think of this concept is in terms of ownership. In ane sense, x owns the memory location. 10 is, at showtime, an empty box that can fit exactly one integer in which integer values tin exist stored.
When you assign a value to 10, you're placing a value in the box that x owns. If you wanted to introduce a new variable (y), you could add this line of code:
This lawmaking creates a new box chosen y and copies the value from ten into the box. Now the retentiveness layout volition look like this:
Detect the new location 0x7f5 of y. Even though the value of x was copied to y, the variable y owns some new accost in memory. Therefore, yous could overwrite the value of y without affecting ten:
Now the memory layout will look similar this:
Again, y'all have modified the value at y, but not its location. In add-on, you accept non affected the original x variable at all. This is in stark contrast with how Python names work.
Names in Python
Python does non have variables. It has names. Aye, this is a pedantic point, and you can certainly use the term variables as much as you like. Information technology is important to know that there is a difference between variables and names.
Let'south take the equivalent code from the above C case and write it in Python:
Much like in C, the above code is broken down into several distinct steps during execution:
- Create a
PyObject - Gear up the typecode to integer for the
PyObject - Set the value to
2337for thePyObject - Create a proper name chosen
x - Point
xto the newPyObject - Increase the refcount of the
PyObjectby i
In retentiveness, it might looks something like this:
You tin see that the retentivity layout is vastly dissimilar than the C layout from earlier. Instead of x owning the block of memory where the value 2337 resides, the newly created Python object owns the retentivity where 2337 lives. The Python name x doesn't directly own whatsoever memory address in the way the C variable ten owned a static slot in memory.
If you were to try to assign a new value to x, y'all could endeavour the following:
What'south happening here is different than the C equivalent, but not also different from the original bind in Python.
This lawmaking:
- Creates a new
PyObject - Sets the typecode to integer for the
PyObject - Sets the value to
2338for thePyObject - Points
xto the newPyObject - Increases the refcount of the new
PyObjectpast i - Decreases the refcount of the onetime
PyObjectpast one
Now in memory, it would look something like this:
This diagram helps illustrate that x points to a reference to an object and doesn't own the memory space as before. It as well shows that the 10 = 2338 command is non an consignment, but rather bounden the proper name x to a reference.
In improver, the previous object (which held the 2337 value) is now sitting in memory with a ref count of 0 and will get cleaned up past the garbage collector.
You could introduce a new proper noun, y, to the mix equally in the C case:
In memory, you would accept a new name, simply not necessarily a new object:
Now you can see that a new Python object has non been created, just a new name that points to the same object. Also, the object's refcount has increased by one. Yous could bank check for object identity equality to ostend that they are the same:
The above code indicates that 10 and y are the same object. Make no mistake though: y is still immutable.
For example, you lot could perform add-on on y:
>>>
>>> y += ane >>> y is x Fake After the improver call, y'all are returned with a new Python object. Now, the retentiveness looks like this:
A new object has been created, and y now points to the new object. Interestingly, this is the aforementioned end-country if y'all had jump y to 2339 directly:
The above argument results in the same end-memory state as the add-on. To recap, in Python, you don't assign variables. Instead, you lot bind names to references.
A Note on Intern Objects in Python
Now that you empathize how Python objects go created and names get bound to those objects, its time to throw a wrench in the machinery. That wrench goes past the proper name of interned objects.
Suppose you accept the post-obit Python code:
>>>
>>> 10 = chiliad >>> y = 1000 >>> x is y Truthful Every bit to a higher place, ten and y are both names that betoken to the same Python object. Only the Python object that holds the value m is non always guaranteed to accept the aforementioned retentiveness accost. For example, if you were to add together two numbers together to get g, yous would cease upwardly with a different retention address:
>>>
>>> x = 1000 >>> y = 499 + 501 >>> x is y False This time, the line ten is y returns False. If this is confusing, then don't worry. Here are the steps that occur when this code is executed:
- Create Python object(
one thousand) - Assign the proper name
tento that object - Create Python object (
499) - Create Python object (
501) - Add together these ii objects together
- Create a new Python object (
1000) - Assign the name
yto that object
Isn't this wasteful? Well, yes it is, but that'due south the price yous pay for all of the great benefits of Python. Y'all never have to worry about cleaning upward these intermediate objects or even demand to know that they exist! The joy is that these operations are relatively fast, and you never had to know any of those details until now.
The core Python developers, in their wisdom, also noticed this waste and decided to make a few optimizations. These optimizations effect in behavior that tin can be surprising to newcomers:
>>>
>>> ten = 20 >>> y = 19 + 1 >>> 10 is y True In this example, y'all run across nearly the same code as earlier, except this time the result is True. This is the result of interned objects. Python pre-creates a certain subset of objects in memory and keeps them in the global namespace for everyday employ.
Which objects depend on the implementation of Python. CPython iii.vii interns the following:
- Integer numbers betwixt
-fiveand256 - Strings that contain ASCII letters, digits, or underscores but
The reasoning behind this is that these variables are extremely likely to be used in many programs. Past interning these objects, Python prevents memory allocation calls for consistently used objects.
Strings that are less than 20 characters and contain ASCII messages, digits, or underscores will exist interned. The reasoning behind this is that these are causeless to be some kind of identity:
>>>
>>> s1 = "realpython" >>> id ( s1 ) 140696485006960 >>> s2 = "realpython" >>> id ( s2 ) 140696485006960 >>> s1 is s2 True Here yous can come across that s1 and s2 both point to the same address in retention. If you were to introduce a not-ASCII letter, digit, or underscore, then yous would get a different result:
>>>
>>> s1 = "Existent Python!" >>> s2 = "Real Python!" >>> s1 is s2 False Because this example has an assertion mark (!) in information technology, these strings are not interned and are different objects in retentivity.
Interned objects are often a source of defoliation. Just call back, if you're ever in doubt, that you can e'er apply id() and is to determine object equality.
Simulating Pointers in Python
But because pointers in Python don't exist natively doesn't mean you tin't go the benefits of using pointers. In fact, in that location are multiple ways to simulate pointers in Python. You lot'll learn 2 in this department:
- Using mutable types every bit pointers
- Using custom Python objects
Okay, allow's get to the betoken.
Using Mutable Types equally Pointers
You've already learned nigh mutable types. Because these objects are mutable, y'all tin can treat them as if they were pointers to simulate arrow behavior. Suppose you wanted to replicate the post-obit c code:
void add_one ( int * ten ) { * x += one ; } This code takes a pointer to an integer (*x) and then increments the value by one. Hither is a master role to exercise the code:
#include <stdio.h> int main ( void ) { int y = 2337 ; printf ( "y = %d \n " , y ); add_one ( & y ); printf ( "y = %d \north " , y ); return 0 ; } In the above code, you assign 2337 to y, print out the current value, increment the value by 1, and and then print out the modified value. The output of executing this code would be the following:
Ane way to replicate this type of beliefs in Python is by using a mutable type. Consider using a listing and modifying the first element:
>>>
>>> def add_one ( ten ): ... x [ 0 ] += 1 ... >>> y = [ 2337 ] >>> add_one ( y ) >>> y [ 0 ] 2338 Here, add_one(x) accesses the outset element and increments its value by ane. Using a listing means that the end result appears to have modified the value. Then pointers in Python practise exist? Well, no. This is only possible because list is a mutable type. If you lot tried to use a tuple, you would get an error:
>>>
>>> z = ( 2337 ,) >>> add_one ( z ) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in add_one TypeError: 'tuple' object does not support item assignment The above code demonstrates that tuple is immutable. Therefore, it does not support particular assignment. listing is non the merely mutable blazon. Another common approach to mimicking pointers in Python is to utilise a dict.
Let's say you lot had an application where you wanted to continue rail of every fourth dimension an interesting outcome happened. One way to achieve this would be to create a dict and use one of the items equally a counter:
>>>
>>> counters = { "func_calls" : 0 } >>> def bar (): ... counters [ "func_calls" ] += 1 ... >>> def foo (): ... counters [ "func_calls" ] += 1 ... bar () ... >>> foo () >>> counters [ "func_calls" ] 2 In this case, the counters dictionary is used to proceed track of the number of function calls. After you lot telephone call foo(), the counter has increased to 2 as expected. All because dict is mutable.
Keep in mind, this is only simulates pointer behavior and does not directly map to true pointers in C or C++. That is to say, these operations are more expensive than they would be in C or C++.
Using Python Objects
The dict choice is a great style to emulate pointers in Python, but sometimes it gets tedious to remember the fundamental name you used. This is particularly true if you're using the dictionary in various parts of your application. This is where a custom Python form can really help.
To build on the concluding case, assume that you desire to runway metrics in your application. Creating a class is a great style to abstract the pesky details:
class Metrics ( object ): def __init__ ( self ): cocky . _metrics = { "func_calls" : 0 , "cat_pictures_served" : 0 , } This code defines a Metrics class. This class still uses a dict for property the actual data, which is in the _metrics member variable. This will give you the mutability you need. Now y'all simply demand to be able to access these values. One overnice way to practise this is with backdrop:
form Metrics ( object ): # ... @property def func_calls ( cocky ): return self . _metrics [ "func_calls" ] @belongings def cat_pictures_served ( self ): return cocky . _metrics [ "cat_pictures_served" ] This lawmaking makes use of @belongings. If you lot're not familiar with decorators, you can bank check out this Primer on Python Decorators. The @property decorator here allows you to admission func_calls and cat_pictures_served as if they were attributes:
>>>
>>> metrics = Metrics () >>> metrics . func_calls 0 >>> metrics . cat_pictures_served 0 The fact that you can access these names as attributes means that you abstracted the fact that these values are in a dict. Yous too go far more explicit what the names of the attributes are. Of course, yous need to be able to increase these values:
class Metrics ( object ): # ... def inc_func_calls ( self ): cocky . _metrics [ "func_calls" ] += 1 def inc_cat_pics ( self ): cocky . _metrics [ "cat_pictures_served" ] += ane You take introduced two new methods:
-
inc_func_calls() -
inc_cat_pics()
These methods change the values in the metrics dict. You now have a class that you alter equally if you're modifying a pointer:
>>>
>>> metrics = Metrics () >>> metrics . inc_func_calls () >>> metrics . inc_func_calls () >>> metrics . func_calls 2 Here, you can access func_calls and call inc_func_calls() in diverse places in your applications and simulate pointers in Python. This is useful when yous have something similar metrics that need to be used and updated frequently in various parts of your applications.
Here's the full source for the Metrics course:
course Metrics ( object ): def __init__ ( self ): self . _metrics = { "func_calls" : 0 , "cat_pictures_served" : 0 , } @property def func_calls ( self ): render self . _metrics [ "func_calls" ] @property def cat_pictures_served ( self ): return cocky . _metrics [ "cat_pictures_served" ] def inc_func_calls ( self ): self . _metrics [ "func_calls" ] += 1 def inc_cat_pics ( self ): self . _metrics [ "cat_pictures_served" ] += i Real Pointers With ctypes
Okay, and so peradventure there are pointers in Python, specifically CPython. Using the builtin ctypes module, you tin create real C-style pointers in Python. If you are unfamiliar with ctypes, so yous tin can take a wait at Extending Python With C Libraries and the "ctypes" Module.
The real reason you would use this is if you needed to make a part telephone call to a C library that requires a arrow. Let'due south get back to the add_one() C-office from earlier:
void add_one ( int * 10 ) { * x += 1 ; } Here again, this code is incrementing the value of x by one. To use this, first compile it into a shared object. Assuming the above file is stored in add together.c, you could accomplish this with gcc:
$ gcc -c -Wall -Werror -fpic add.c $ gcc -shared -o libadd1.and then add.o The showtime command compiles the C source file into an object called add.o. The second control takes that unlinked object file and produces a shared object called libadd1.so.
libadd1.so should be in your electric current directory. You can load it into Python using ctypes:
>>>
>>> import ctypes >>> add_lib = ctypes . CDLL ( "./libadd1.so" ) >>> add_lib . add_one <_FuncPtr object at 0x7f9f3b8852a0> The ctypes.CDLL code returns an object that represents the libadd1 shared object. Because yous divers add_one() in this shared object, you can admission it as if it were any other Python object. Before you call the function though, y'all should specify the function signature. This helps Python ensure that yous pass the right type to the office.
In this example, the function signature is a pointer to an integer. ctypes will allow you to specify this using the following lawmaking:
>>>
>>> add_one = add_lib . add_one >>> add_one . argtypes = [ ctypes . Arrow ( ctypes . c_int )] In this code, you're setting the office signature to match what C is expecting. Now, if you were to effort to phone call this lawmaking with the wrong type, and then you would get a nice warning instead of undefined behavior:
>>>
>>> add_one ( 1 ) Traceback (well-nigh recent call last): File "<stdin>", line i, in <module> ctypes.ArgumentError: argument 1: <form 'TypeError'>: \ expected LP_c_int instance instead of int Python throws an mistake, explaining that add_one() wants a pointer instead of just an integer. Luckily, ctypes has a mode to pass pointers to these functions. Showtime, declare a C-style integer:
>>>
>>> x = ctypes . c_int () >>> x c_int(0) The above lawmaking creates a C-way integer x with a value of 0. ctypes provides the handy byref() to allow passing a variable by reference.
Y'all can use this to call add_one():
>>>
>>> add_one ( ctypes . byref ( x )) 998793640 >>> ten c_int(one) Nice! Your integer was incremented by one. Congratulations, you take successfully used real pointers in Python.
Decision
You now have a better understanding of the intersection between Python objects and pointers. Even though some of the distinctions between names and variables seem pedantic, fundamentally agreement these key terms expands your agreement of how Python handles variables.
You lot've also learned some excellent ways to simulate pointers in Python:
- Utilizing mutable objects as low-overhead pointers
- Creating custom Python objects for ease of use
- Unlocking real pointers with the
ctypesmodule
These methods allow you to simulate pointers in Python without sacrificing the retention rubber that Python provides.
Thank you for reading. If you still have questions, experience free to reach out either in the comments department or on Twitter.
Spotter Now This tutorial has a related video course created past the Real Python squad. Watch it together with the written tutorial to deepen your understanding: Pointers and Objects in Python
garfieldbuttephon.blogspot.com
Source: https://realpython.com/pointers-in-python/
0 Response to "If You Create an Objects Thats a Pointer and Then Set It to New Again What Happens"
Post a Comment