Python Multiprocessing Shared Object

Python Multiprocessing Shared Object

  1. Use Python Shared Memory Objects in Multiprocessing
  2. Conclusion

In Python, shared memory multiprocessing is made up of connecting multiple processors, but these processors must have direct access to the system’s main memory. This will allow all connected processors to access the other processor data they have used or created.

Use Python Shared Memory Objects in Multiprocessing

Using multiprocessing in Python, a new process can run independently and have its own memory space. By seeing the examples below, let’s understand shared object multiprocessing using Python in detail.

Example Code:

import multiprocessing

#an empty array globally declared
answer = []

def square_numbers(mynumbers):

#for squaring array elements, a function has been used

    global answer
    #appending square numbers to a global array
    for n in mynumbers:
        answer.append(n * n)
    #print a global array for generating an answer
    print("Answer using first process: {}".format(answer))

if __name__ == "__main__":
    #input array
    mynumbers = [5,10,15]

    #new process has been created
    p = multiprocessing.Process(target=square_numbers, args=(mynumbers,))
    #process begins here
    p.start()
    #wait unless a process is completed
    p.join()

    #print a global array for generating an answer
    print("Answer using main program: {}".format(answer))

Output:

Answer using first process: [25, 100, 225]
Answer using main program: []

We’ve printed global array answers in two places using the above example.

The process p is called the square_numbers function so that the array elements would be changed for process p in the memory space.

The main program is run after process p completes, and we will get the empty array as an answer in the memory space.

Multiprocessing in Python provides value objects and an array for sharing data between multiple processes.

Example Code:

import multiprocessing

def square_data(mydata, answer, square_sum):
  #a function has been made for squaring of given data

    #appending squares of mydata to the given array
    for ix, n in enumerate(mydata):
        answer[ix] = n * n

    #sum the square values
    square_sum.value = sum(answer)

    #print array of squared values for process p
    print("Answer in process p: {}".format(answer[:]))

    # print the sum of squared values for process p
    print("Sum of squares values in process p: {}".format(square_sum.value))

if __name__ == "__main__":
    #here, we input the data
    mydata = [1,2,3]

    #an array has been created for the int data type for three integers
    answer = multiprocessing.Array('i', 3)

    #value has been created for int data type
    square_sum = multiprocessing.Value('i')

    #new process has been created
    p = multiprocessing.Process(target=square_data, args=(mydata, answer, square_sum))

    #process begins from here
    p.start()

    #wait unless the process is completed
    p.join()

    # print an array of squared values for the main program
    print("Answer in main program: {}".format(answer[:]))

    # print the sum of squared values for the main program
    print("Sum of square values in main program: {}".format(square_sum.value))

Output:

Answer in process p: [1, 4, 9]
Sum of squares in process p: 14
Answer in main program: [1, 4, 9]
Sum of squares in main program: 14

In the above example, we’ve created an array and passed three integers to it. We’ve printed an array of squared values and then a sum of square values for process p.

After this, we again printed an array of squared values and the sum of square values for the main program.

Conclusion

There could be several ways to explain shared memory multiprocessing using Python. So, in this article, we’ve explained the multiprocessing shared memory concept that how an object could be placed in shared memory space and run independently.

Apart from this, we have also learned that Python allows processes to share data between various processes.

Zeeshan Afridi avatar Zeeshan Afridi avatar

Zeeshan is a detail oriented software engineer that helps companies and individuals make their lives and easier with software solutions.

LinkedIn

Related Article - Python Multiprocessing

  • Python Multiprocessing Logging
  • Multiprocessing vs Threading in Python
  • Pool Map With Multiple Arguments in Python