How to Lock a File in Python
- Impact of Resource Sharing by Multiple Processes in Python
- File State and Its Impact on Locking a File in Python
- Lock a File in Python
- Conclusion
Some resources cannot be accessed by two people simultaneously in our daily life, for example, a changing room. Every changing room has a lock that only one person can access at a time.
If two people had access simultaneously, it could cause confusion and embarrassment. So, that’s why the shared resource is protected with a lock.
Similarly, in programming, whenever two processes or threads share the same resource, it can create problems that must be avoided by using locks. This article will explain how to lock a file in Python.
Impact of Resource Sharing by Multiple Processes in Python
This section will clarify why locking a file in Python is important.
The program has two processes: the first deposits $1 in an account, another deducts $1 from it, and then the balance is printed in the end. This operation is run inside a loop thousands of times.
Technically, the iteration would result in no change in the final balance as the same number is added and deducted over and over. But when the program is compiled, the results are variable.
import multiprocessing
# Withdrawal function
def wthdrw(bal):
for _ in range(10000):
bal.value = bal.value - 1
# Deposit function
def dpst(bal):
for _ in range(10000):
bal.value = bal.value + 1
def transact():
# initial balance
bal = multiprocessing.Value("i", 100)
# creating processes
proc1 = multiprocessing.Process(target=wthdrw, args=(bal,))
proc2 = multiprocessing.Process(target=dpst, args=(bal,))
# starting processes
proc1.start()
proc2.start()
# waiting for processes to finish
proc1.join()
proc2.join()
# printing final balance
print("Final balance = {}".format(bal.value))
if __name__ == "__main__":
for _ in range(10):
# Performing transaction process
transact()
Output:
C:\python38\python.exe "C:\Users\Win 10\main.py"
Final balance = 428
Final balance = 617
Final balance = -1327
Final balance = 1585
Final balance = -130
Final balance = -338
Final balance = -73
Final balance = -569
Final balance = 256
Final balance = -426
Process finished with exit code 0
Let’s break down the example code to understand why it happens.
This program uses the Python library package multiprocessing to invoke methods as processes in the program.
The two methods, wthdrw and dpst, deduct and add money to the account. They have one parameter, bal, which stands for balance.
Inside the method, the value of bal is incremented or decremented inside a for loop. The iteration is repeated 10000 times.
The bal.value is the shared resource in this program.
# Method to Decrement
def wthdrw(bal):
for _ in range(10000):
bal.value = bal.value - 1
A new method, transact(), launches the processes one over the other. Inside the method, the object bal, which stores the initial value, is created, and the balance is set as 100.
def transact():
bal = multiprocessing.Value("i", 100)
lock = multiprocessing.Lock()
The multiprocessing library allows the launching of methods as processes using objects. For example, the method wthdrw is launched by creating a process proc1.
This process targets the wthdrw method and passes bal and an empty object as arguments.
proc1 = multiprocessing.Process(target=wthdrw, args=(bal,))
Similarly, a proc2 process is created to launch the dpst method. Once both processes are created, they are initiated using process_name.start().
proc1.start()
proc2.start()
To close a running process, the process_name.join() syntax is used. It queues the request behind a running process, so the system waits for the process to get finished and then closes it.
The final balance is printed after the process is completed.
print("Final balance = {}".format(bal.value))
The __name__ variable is set to __main, and the method transact() is called inside a for loop. This lets us observe the iterations multiple times without manually calling processes.
if __name__ == "__main__":
for _ in range(10):
transact()
When the program was run, we found that the values were inconsistent. The shared resource bal.value is not protected with a lock, so the other process edits the value before the running process has finished.
Locks are used to solve this problem. The above program is locked by placing a lock in both methods; this way, the compiler has to wait to run the next process before the previous one is complete.
# Withdrawal function
def wthdrw(bal, lock):
for _ in range(10000):
# Locks
lock.acquire()
bal.value = bal.value - 1
lock.release()
The lock.acquire() statement locks the process, and inside it, the value of bal is edited. After it, the lock is released using lock.release().
The same is applied to the dpst method, and the rest of the program remains the same.
Output:
C:\python38\python.exe "C:\Users\Win 10\main.py"
Final balance = 100
Final balance = 100
Final balance = 100
Final balance = 100
Final balance = 100
Final balance = 100
Final balance = 100
Final balance = 100
Final balance = 100
Final balance = 100
Process finished with exit code 0
As we know what happens to multiple processes with shared resources, we will look at an important functionality needed to lock a file in Python - File State.
File State and Its Impact on Locking a File in Python
A file state shows if the file is open or closed. It is important to know the file state to lock a file in Python because locking operations can only be placed on open files.
In Python, suppose there is a file - myFile.txt. When we write:
readMyFile = open("myFile.txt", "r").read()
It is impossible to get the state of the file, close it or do anything else. There is no access because it is not stored anywhere.
Since there would be no value in discarding information and the impossibility of garbage collection, there is no magic that allows the retrieval of knowledge that has been lost.
Refrain from storing any value intended to be used in a variable (a list, a dict member, or an attribute of an instance), which will allow it to be used.
myFile = open("myFile.txt", "r")
readMyFile = myFile.read()
print(myFile.closed)
myFile.close()
print(myFile.closed)
The above code saves the file in a variable myFile. Then another variable, readMyFile, uses the myFile variable to read the object.
Using this method keeps the file in an open state even after the reading is complete.
The syntax print(myFile.closed) shows the current file state; here, it returns a false value, which means it is not closed. This file is explicitly closed using myFile.close().
As explicitly closing files are prone to cause human errors, a better solution is to use the with statement.
with open("myFile.txt", "r") as myFile:
readMyFile = myFile.read()
Now, myFile gets automatically closed without the need to close it explicitly. From the above example, we understood that efficient methods in opening and closing files are essential to lock a file in Python.
A file can only be locked in an open state, and a file cannot be unlocked when there’s no lock present. Locking a file in Python must always be executed inside a with block.
Let’s look at several ways to lock a file in python.
Lock a File in Python
File locks are generally third-party libraries used inside scripts to lock a file. In this section, some widely used file locks are discussed.
Lock a File Using FileLock in Python
This is a platform-specific file-locking library that works both on Windows and UNIX. To install it, type the command:
pip install filelock
Locking a file with this method is very simple. Open the file with FileLock inside a with statement, so the file opens, and the system locks it.
After the process is complete, the lock is released automatically at the end of the with statement.
from filelock import FileLock
with FileLock("myfile.txt"):
print("Lock acquired.")
Output:
C:\python38\python.exe "C:/main.py"
Lock acquired.
Process finished with exit code 0
Another way to lock a file in Python is with timeouts. This helps put a time limit if a file cannot be locked, thus releasing file handles for other processes.
from filelock import FileLock
file = "example.txt"
lockfile = "example.txt.lock"
lock = FileLock(lockfile, timeout=5)
lock.acquire()
with open(file, "a") as f:
print("File Locked")
f.write("Add some data \n")
lock.release()
with open(file, "a") as f:
f.write("Second time writing \n")
The first line of code imports the FileLock module from the filelock library. An example.txt file is saved inside a variable file; this file will be used to write on it.
The lock is not directly placed on the original file, so a temporary file is created, like example.txt.lock. The variable lock is used to create a lock object for the lockfile, and a timeout of 5 seconds is set.
This lock can be placed using the lock.acquire statement. A with block is used to open the file and write to it, while the lockfile avoids any other process to access the original file while it is being written.
Finally, the lock is released using lock.release(). Another process then opens the file and successfully writes on it.
Output:
C:\python38\python.exe "C:/main.py"
File Locked
Process finished with exit code 0
To lock a file using Python, we can also place the locks in a nested manner:
from filelock import Timeout, FileLock
lock = FileLock("high_ground.txt.lock")
with lock:
with open("high_ground.txt", "a") as f:
f.write("You were the chosen one.")
In the above example, a lock object is created with a file named high_ground.txt to lock a file in Python. The lock is placed inside a block; inside it, the file is read using another block.
This method requires less code than the previous one.
The FileLock is platform-dependent. If every application instance operates on the same platform, use a FileLock; otherwise, use a SoftFileLock.
A SoftFileLock is independent of platforms and only monitors the lock file’s existence. Because of this, it is extremely portable and more likely to freeze up if the application crashes.
In such circumstances, delete the lock file.
File locker can also be used with an exception handling block:
try:
with lock.acquire(timeout=10):
with open(file_path, "a") as f:
f.write("I have a bad feeling about this.")
except Timeout:
print("Another instance of this application currently holds the lock.")
This code snippet places a lock inside the try block and gives a timeout for 10 seconds. Then inside a nested with, the file is written.
Inside the except block, a suitable message is set to be printed if the application gets timed out.
Lock a File Using PortaLocker in Python
Another option to lock a file in Python is using the library called portalocker, which offers a simple API for file locking.
It’s crucial to remember that locks are advisory by default on Linux and Unix systems. Obligatory file locking can be enabled on Linux by adding the -o mand option to the mount command.
To install it, type the command:
pip install "portalocker[redis]"
Locking a file in Python using portalocker is similar to the FileLock method but much simpler.
import portalocker
file = "example.txt"
with portalocker.Lock(file, "a") as fh:
fh.write("first instance")
print("waiting for your input")
input()
Output:
C:\python38\python.exe "C:/main.py"
lock acquired
waiting for your input
In the above code, the first line imports the library package. The lock is created using the syntax portalocker.RedisLock(), and the lock is placed using a with statement.
To place the lock inside a timeout, use:
import portalocker
file = "example.txt"
with portalocker.Lock("file", timeout=1) as fh:
print("writing some stuff to my cache...", file=fh)
Output:
Conclusion
This article explains how to lock a file in Python and why it is important. The reader can create open-state files and put locks on them easily after reading this article.