Today in this example we will learn how to perform Python threading with a few examples and we will also learn the Best Practices.

We will see how to optimize code execution by performing Python asynchronous code execution.

We will cover the below aspects in today’s article,

Asynchronous programming is a programming allows tasks to be executed concurrently and independently of each other, without blocking the execution of the main program. In asynchronous programming, tasks are executed asynchronously, meaning they can start, run, and complete independently of the main program’s execution flow.

The key concept in asynchronous programming is the use of non-blocking operations, where a program can initiate a task and continue executing other tasks without waiting for the completion of the initiated task. This enables better utilization of resources and improved responsiveness in applications, especially when dealing with I/O-bound operations such as network requests, file I/O, and database queries.

Getting Started – Multithreading with synchronous(Blocking) and asynchronous(non-bloking) execution

Before we deep dive into the Python multithreading example, please note that there are multiple ways one can obtain Multithreaded execution in Python.

Main techniques inlcues like using Python Threading vs ThreadPoolExecutor Vs Asynco Python.

Let’s take a simple use case using a Python application where I have one operation that needs to be Multithreading with parameters in Python.

I have a requirement to invoke multiple service calls and combine their decisions to display to the user as if like a single operation.

However, the below concept discussed is generic enough and can be applied to any Python business logic as well.

I have the below IDs to be processed asynchronously.

Each of above the IDs needs to be processed by calling an HTTP API method.

The below API processes the above IDs and provides the result,

Let’s now see how to launch each request sequentially and parallelly and achieve the result.

Old – Executing Synchronous method in Python

I have a simple example where a service request takes a few seconds to execute.

# Define an synchronous function to be executed
def perform_task(parameter):
    # Start time
    start_time = time.time()
    print(f"Task executing with parameter: {parameter}")

     # Simulate some asynchronous work by calling Sleep 
    time.sleep(5) 

     # End time
    end_time = time.time()
    elapsed_time = end_time - start_time
    print(f"Operation took {elapsed_time} seconds")
    print("Task execution complete")

Since we are executing the method synchronously, each operation executes in order and one after the other.

Below is the main function

def main():
    # Define the parameters
    parameters = ["param1", "param2", "param3"]
    # Start time
    start_time = time.time()
    # Create and gather tasks for each parameter
     # Perform the task for each parameter
    for param in parameters:
        perform_task(param)
      # End time
    end_time = time.time()
    elapsed_time = end_time - start_time
    print(f"Task execution complete . Total Time taken : {elapsed_time} seconds")

if __name__ == "__main__":
      main()

So short, each call is a blocking call and takes its own time before proceeding to the other.

Each synchronous operation executes, so the total time required to complete the operation is equal to the sum of all requests processed operations.

The total time required to complete all operations is ~ 15 second

Below is the code base for Synchronous operation,

Asynchronous Task execution in Python using Asyncio

Please add below import statement below in your Python file

import asyncio
import time

Let’s declare the above operation using the asyncio Python example

A main() method as below,

async def main():
    # Define the parameters
    parameters = ["param1", "param2", "param3"]
    # Start time
    start_time = time.time()
    # Create and gather tasks for each parameter
    tasks = [perform_task(param) for param in parameters]
    await asyncio.gather(*tasks)
      # End time
    end_time = time.time()
    elapsed_time = end_time - start_time
    print(f"Task execution complete . Total Time taken : {elapsed_time} seconds")

if __name__ == "__main__":
    asyncio.run(main())

In this example:

• We define an asynchronous method perform_task that represents the asynchronous task to be executed by each coroutine.
• We use asyncio.gather() to run all coroutines concurrently.
• We used asyncio.run() to run the main coroutine.
• We measure the total time taken for the execution by recording the start time before running the main coroutine and the end time after it completes. Then, we calculate and print the elapsed time.

Finally, the result is displayed below,

Each operation executes parallel and the maximum time taken by any operation would be the actual time taken by the whole operation.

So all the method execution gets completed or content around the same time as one of the operations.

However, timing may vary depending on CPU, CORE, and RAM memory GB size as needed.

Increasing the load

Now even if I increase the request load by putting more request execution you will see the total time required would remain near the same time of execution and allowing us to get better throughout overall.

However, as highlighted above this also depends on the CPU load at the time of code execution and also depends on the CPU used by the machine, and its hardware configuration

Note- Use must use async and await throughout your codebase consistently get the full benefit of asynchronous execution.

Do you have any comments or ideas or any better suggestions to share?

Please sound off your comments below.

Happy Coding !!

Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published (2025) best practices and guidelines for software design and development.

The post Python Threading and Asynchronous Task examples – Best Practices first appeared on TheCodeBuzz.

Today in this article, we will see how to use PySpark Connect to MongoDB using Python code examples.

We will make use of MongoDB Spark Connector which helps connect to MongoDB and allows us to read and write data between PySpark and MongoDB.

Here’s a basic example demonstrating how to read data from MongoDB, perform a transformation, and then write the results back to MongoDB.

We have below MongoDB collection,

Install the MongoDB Spark Connector

Please install MongoDB Spark Connector using the below command.

Command

pip install pyspark

For more details, please visit the official MongoDB website:

https://www.mongodb.com/products/spark-connector

Create a SparkSession Object

Once you have PySpark running, you can use the below command to create a PySpark session object as below,

from pyspark.sql import SparkSession


my_spark = SparkSession 
    .builder 
    .appName("TheCodeBuzz") 
    .config("spark.mongodb.input.uri", "mongodb://host/TheCodeBuzz.Orders") 
    .config("spark.mongodb.output.uri", "mongodb://host/TheCodeBuzz.Orders") 
    .getOrCreate()

We create a Spark session with the MongoDB connector configurations.

We will use the above SparkSession object to read, and write data to MongoDB, etc.

Read Data from MongoDB using PySpark

You must specify the following configuration settings to read from MongoDB:

dataFrame = spark.read
                 .format("mongodb")
                 .option("database", "TheCodeBuzz")
                 .option("collection", "Orders")
                 .load()

Above spark.read() function returns a DataFrameReader object, which you can use to specify the format and other configuration settings for batch read operation.

Write data to MongoDB using PySpark

To write data from MongoDB, call the write function on your SparkSession object.

You must specify the following configuration settings to write the data to MongoDB,

data = [
  {

    "Order": "journal2",
    "qty": 25,
    "books": [
      "white"
    ],
    "domain": [
      14
    ],
    "Link": "https://www.thecodebuzz.com/order/23456"
  },
  {

    "Order": "journal1",
    "qty": 25,
    "books": [
      "black"
    ],
    "domain": [
      15
    ],
    "Link": "https://www.thecodebuzz.com/order/2324"
  }
]

dataFrame = spark.read.json(sc.parallelize([data]))


dataFrame.write.format("mongodb")
               .mode("append")
                .option("database", "TheCodeBuzz")
                .option("collection", "Orders")
                .option("upertDocument", "true")
                .save()

The above example createDataFrame() function creates a DataFrame object from JSON input and saves it to MongoDB.

That’s all! Happy coding!

Does this help you fix your issue?

Do you have any better solutions or suggestions? Please sound off your comments below.

Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published (2025) best practices and guidelines for software design and development.

The post Python – PySpark Connect to MongoDB first appeared on TheCodeBuzz.

Today this article will learn to write query using Python MongoDB – Replace String in Pymongo with example.

We will replace a string in the Mongo document with the input string using a simple and easy approach i.e using $replaceOne aggregation operator using Pymongo driver code.

We will cover below aspects,

We already looked at a simple way of adding or updating a new field to the document in our previous MongoDB sample series.

Getting started – Pymongo

Here below is a sample schema or document we shall use,

Python Replace Substring in MongoDB – $replaceOne Aggregator

We are going to use a newly introduced aggregate operator to replace easily a string with matching criteria with part of the string.

PyMongo- Query Pattern using $replaceOne

{ 

$replaceOne: { 
    input: <expression>, 
    find: <expression>, 
    replacement: <expression>}
}

In the above query,

You can replace the existing Mongo field using any other fields from the document using the above approach + using the approach discussed here.

Example using $replaceOne

In the below example, we will replace “total-order” with “order” in the mongo field.

Please add below using the namespace in your code,

import re

from pymongo import MongoClient

For more details on Pymongo installation and details, please visit https://api.mongodb.com/python/current

Define PyMongo Replace query

Let’s define the filter using $replaceOne as below,

filter= {
    'Link': {
        '$regex': re.compile(r"total-order")
    }
},[
    {
        '$set': {
            'Link': {
                '$replaceOne': {
                    'input': '$Link', 
                    'find': 'total-order', 
                    'replacement': 'order'
                }
            }
        }
    }
]

Above we are defining regex to find all the document where string matches with “total-order”.

This match will filter only records with matching criteria and hence fasten the search.

Below is the complete code with example?

import re
from pymongo import MongoClient

client = MongoClient('your connection string')

filter= {
    'Link': {
        '$regex': re.compile(r"total-order")
    }
},[
    {
        '$set': {
            'Link': {
                '$replaceOne': {
                    'input': '$Link', 
                    'find': 'total-order', 
                    'replacement': 'order'
                }
            }
        }
    }
]

result = client['TheCodeBuzz']['Orders'].find(
  filter=filter
)

Below is the schema post-update,

Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published (2025) best practices and guidelines for software design and development.

The post Python MongoDB – Replace String in Pymongo with example first appeared on TheCodeBuzz.

Today in this article, we will see how to perform PyMongo – Insert datetime as ISODate in Python.

We will use Mongo Driver called pymongo to insert the date in the collection in the ISO format.

Getting started

Here below is a sample schema or document we shall use for the date range query,

Install PyMongo package

Please install the PyMongo driver package to get support for the MongoClient connection class and properties.

For more details visit- https://api.mongodb.com/python/current

Add PyMongo Import statements

Please add below using namespaces,

from pymongo import MongoClient

from datetime import datetime

If you need support for date time zone support in UTC, ETC, etc, please add below other import statements.

from datetime import datetime, tzinfo, timezone

Define MongoDB Connection

client = MongoClient('mongodb://localhost:27017/')
db = client['your_database']
collection = db['your_collection']

Python – Insert Date Time in ISO format

Define the date for the query

date = datetime(2023, 1, 1)

OR

Date time using time zone

date = datetime(2023, 1, 1, tzinfo=timezone.utc)

Note: When used with datetime, MongoDB by default uses ISO format to insert any date in the mongo collection.

Define the filter

filter =  { //define any filter as needed }

Define the update operation

Please define the update operation (update_operation) to fit your data structure and update requirements as needed.

Below we are setting the “Date” the date field with ISO date in all the documents in the collection.

# Define the update operation

update_operation = {
    '$set': {
        'ModifiedDate': date
    }
}

Perform update_many to insert the ISO date

Let’s perform the update_many operations as below. update_many in PyMongo permits you to update multiple documents that match a specified filter.

We simplification purposes, we will insert the ISO date in every document using update_many.

result = collection.update_many({},update_operation)

Above I have defined an empty filter.

Do you have any comments or ideas or any better suggestions to share?

Please sound off your comments below.

Happy Coding !!

Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published (2025) best practices and guidelines for software design and development.

The post PyMongo – Insert datetime as ISODate in Python first appeared on TheCodeBuzz.

Today in this article we will learn Python RSA Key pair Encryption and Decryption with examples.

We will use a Python package called PyMySQL and configure the connection parameters to include the public key for encryption and the private key for decryption.

PyMySQL supports connecting to a MySQL server using RSA keys for secure communication.

Here’s how you can use RSA keys with PyMySQL.

Generate RSA key Pair – public and private keys

First, we will generate an RSA key pair (public and private keys).

Lets the OpenSSL library generate the keys.

Open a terminal or command prompt and run the following commands.

Generate Private Keys

Let’s create a private key with a 2048-bit key length.

openssl genpkey -algorithm RSA -out privatekey.pem -pkeyopt rsa_keygen_bits:2048

Generate Public Keys

openssl rsa -pubout -in privatekey.pem -out publickey.pem

Install PyMySQL with RSA

Please install the cryptography package,

pip install cryptography

Use RSA keys in your Python code

Kindly add below import statements

import pymysql.cursors 
from cryptography.hazmat.backends import default_backend 
from cryptography.hazmat.primitives import serialization

Read RSA keys in Python code

Let’s read files from the files using the below code,

Read private keys

with open("privatekey.pem", "rb") as f:
    private_key = serialization.load_pem_private_key(
        f.read(), password=None, backend=default_backend()
    )

Read public keys

with open("publickey.pem", "rb") as f:
    public_key = serialization.load_pem_public_key(
        f.read(), backend=default_backend()
    )

Connect using RSA keys with PyMySQL

Connect to the database

connection = pymysql.connect(host='localhost',
                             user='user',
                             password='passwd',
                             database='db',
                             cursorclass=pymysql.cursors.DictCursor,
                             "ssl": {
        "ca": "/path/../server-ca.pem",  # Path Server CA certificate (optional)
        "cert": "/path/../client-cert.pem", # Path client certificate(optional)
        "key": "/path/../client-key.pem",  # Path private key (optional)
        "cipher": "TLSv1.2",
        "keyfile": public_key,  # Use public key for encryption
        "certfile": private_key,  # Use private key for decryption
    })

In the above,

The ssl property includes the paths to the server CA certificate, client certificate, and client private key.

It uses the RSA keys for encryption and decryption by setting the keyfile and certfile to the respective loaded keys.

Read from the database

Let’s perform read operations on the MySQL Database,

  with connection.cursor() as cursor:
        # Read a single record
        sql = "SELECT `College`, `Name` FROM `Employee` WHERE `ID`=%s"
        cursor.execute(sql)
        result = cursor.fetchone()
        print(result)

Do you have any comments or ideas or any better suggestions to share?

Please sound off your comments below.

Happy Coding !!

Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published (2025) best practices and guidelines for software design and development.

The post Python RSA Key pair Encryption and Decryption with examples first appeared on TheCodeBuzz.