Raspberry Pi Zero 2W : How to log data to an external data service easily like MongoDB

In this example we'll learn how to set things up so that we can send data to external service and save it there reliably for long tern storage solution, rather than storing data locally with space and size constraints.

So, lets get started without wasting any time.

What is Raspberry Pi ?

It's a small, affordable, single-board computer developed by the Raspberry Pi Foundation. Designed initially for educational purposes, it provides a platform for learning programming, computer science, and even in few industries. Over time, it has gained popularity among hobbyists and professionals for its versatility in various projects, such as home automation, media centers, robotics, and more.

The Raspberry Pi runs a variety of operating systems, most notably Raspberry Pi OS (formerly Raspbian), which is based on Debian Linux. It supports programming languages like Python, C, C++, Java, and others, making it an ideal tool for both beginners and advanced users.

The device features USB ports, HDMI output, GPIO pins for hardware projects, and network connectivity, making it a powerful and flexible tool for numerous applications.

Pi Zero 2W

The Raspberry Pi Zero 2W is a compact and cost-effective variant of the Raspberry Pi series, designed to offer significant performance improvements over its predecessor, the Raspberry Pi Zero W.

Released by the Raspberry Pi Foundation, the Pi Zero 2W retains the small form factor of the original Pi Zero models but its new features and enhanced specifications, making it more capable for various projects.

Features

• A quad-core 64-bit ARM Cortex-A53 CPU, which provides a significant performance boost compared to the single-core CPU of the original Pi Zero.
• 512MB of LPDDR2 SDRAM, sufficient for lightweight computing tasks and various IoT applications.
• Integrated 2.4GHz IEEE 802.11b/g/n wireless LAN and Bluetooth 4.2, offering robust wireless communication capabilities.
• CSI-2 camera connector, allowing the attachment of Raspberry Pi camera modules.
• Mini HDMI port for video output.

The Raspberry Pi Zero 2W is designed to be a low-cost, low-power solution suitable for projects that require a small form factor and modest computing power. Its improved performance and connectivity make it ideal for embedded applications, DIY electronics, portable projects, and educational purposes.

External Data Logging

For the purpose of logging data, we'll  be using Mongo DB for easy and reliable storage of data. Why? Because there are many good libraries to help us with the process and not to mention the community support. 

What is MongoDB ?

MongoDB is a popular NoSQL database that uses a document-oriented data model. Unlike traditional relational databases that use tables and rows, MongoDB stores data in flexible, JSON-like documents.

Here are some key features and concepts associated with MongoDB:

• Document-Oriented: Data is stored in documents (similar to JSON objects), which allows for flexible and dynamic schemas. This means each document can have different fields and structures, making it easier to store complex hierarchical data.
• Collections: Documents are grouped into collections, which are analogous to tables in relational databases. However, unlike tables, collections do not enforce a fixed schema.
• Schema Flexibility: MongoDB's schema-less design allows for rapid iteration and changes to the data model without requiring extensive database migrations.
• Scalability: MongoDB is designed to scale horizontally by distributing data across multiple servers (sharding). This makes it suitable for handling large volumes of data and high-throughput operations.
• Indexing: MongoDB supports various types of indexes to optimize query performance, including single-field, compound, and geospatial indexes.
• Aggregation Framework: MongoDB provides a powerful aggregation framework for performing data analysis and transformations. It allows for operations like filtering, grouping, sorting, and reshaping data.
• Replication: MongoDB offers replica sets, which are groups of MongoDB instances that maintain the same data set, providing high availability and data redundancy.
• ACID Transactions: Starting from version 4.0, MongoDB supports multi-document ACID (Atomicity, Consistency, Isolation, Durability) transactions, making it possible to perform complex operations across multiple documents with consistency guarantees.
• High Performance: MongoDB is optimized for fast read and write operations, making it suitable for applications that require low-latency access to data.
• Wide Adoption: MongoDB is widely used in various industries, including finance, gaming, IoT, e-commerce, and more, due to its versatility and ease of use.

In short, MongoDB is a highly flexible and scalable database that is well-suited for modern applications requiring dynamic data models, high performance, and ease of scalability.

MongoDB Atlas

MongoDB Atlas is a fully managed cloud database service provided by MongoDB. It simplifies the deployment, management, and scaling of MongoDB databases in the cloud.

Here are some of its key features and benefits:

• Fully Managed Service: MongoDB Atlas takes care of database administration tasks such as provisioning, patching, scaling, and backups, allowing developers to focus on building applications rather than managing infrastructure.
• Multi-Cloud Support: Atlas runs on multiple cloud platforms, including Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure. This allows users to choose the cloud provider that best fits their needs and take advantage of multi-cloud strategies.
• Scalability: Atlas supports horizontal scaling through sharding, allowing databases to handle large datasets and high-traffic workloads. It also offers vertical scaling by upgrading instance sizes as needed.
• Performance Optimization: Atlas provides automated performance optimization tools, such as index recommendations and query performance insights, to help maintain efficient database operations.
• Monitoring and Alerts: Atlas provides comprehensive monitoring and alerting capabilities, allowing users to track database metrics and receive notifications for potential issues.
• Serverless and Dedicated Options: Atlas offers flexible deployment options, including serverless instances for applications with unpredictable workloads and dedicated clusters for more consistent performance and resource control.
• Backup and Restore: Atlas offers automated backup services with point-in-time recovery options, ensuring that data can be restored to any specific moment within the backup retention period.

Hardware Requirements of this Project

• Raspberry Pi Zero 2W
• Pi Zero/ Zero 2W Case
• 32GB Micro SD Card
• Heat Sinks
• Cables

Software Requirements of this Project

• Python Packages and few other dependencies

Prerequisites

Before we begin, we gonna need a fully ready Pi Zero 2W. If you don't have one yet, please visit this guide here. And follow the steps to get the Pi Zero 2W ready in no time.
In there you'll also find a guide on how to SSH into your Pi so that you don't need extra display and inputs. 

Now we need to install few libraries before we can run code on it.
1. dotenv for accessing sensitive information securely like password stored on the system.
2. motor for accessing MongoDB, as motor provides asynchronous functionalities.
3. asyncio to make our code asynchronous, mitigating any deadlocks or slowdowns.

Install the above libraires by the command-
sudo apt install python3-dotenv
sudo apt install python3-motor
sudo apt install python3-asyncio

After installing the above libraires, next we move onto making a folder where we'll save the files required for this project.
Make a folder by the command - mkdir database_examples
Now we move into that folder by cd database_examples/

Now first we head into the MongoDB's website and make a new project there and get a cluster allocated. For this well be needing the free tier which is shared option. However the method will be the same for any of the tiers.

After getting a cluster, head into its database access. Make a new user, and set its permissions. And generate the password. Copy that password, we gonna need it later. Now save and exit that page.
Create a database, give its name for eg: Test and deploy it. It might take some time.

Coming back to the console, now we create a file in that folder named token.env by -
sudo nano token.env,  and in here we'll be storing all of our passwords and and other sensitive information's.  
username = <the name of the user>
password = <the password you copied before hand>
and save and exit the editor.

Next we make out main file by the command sudo nano test.py for example. And copy the code from the coding section and past it there. I have included all the basic functions in one file, and you can study the code and modify it to suit your need.

Code

Copy the following code, and save it. And after exiting the editor, run it by command python <file_name>.py

from dotenv import dotenv_values
from time import sleep
import asyncio
import motor.motor_asyncio

cluster = motor.motor_asyncio.AsyncIOMotorClient("mongodb+srv://"+dotenv_values('token.env')['username']+":"+dotenv_values('token.env')['password']+"@test.7sf9cvi.mongodb.net/?retryWrites=true&w=majority&appN>db = cluster["Shop"]
collection = db["Shelfs"]

async def find_one():
        reply = await collection.find_one({"_id": 1})
        if reply:
                print("Find Success")
                print(reply)
        else:
                print("Find Failiure")

async def insert_one():
        reply = await collection.insert_one({"_id": 1, "Name": "Hello"})
        if reply:
                print("Insert Success")
                print(reply)
        else:
                print("Insert Failiure")

async def update_one():
        update = await collection.update_one({"_id": 1},{"$set": {"Name": "Hello World"}})
        if update:
                print("Update Success")
                print(update)
        else:
                pritn("Update Failiure")

async def delete_one():
        reply = await collection.delete_one({"_id": 1})
        if reply:
                print("Delete Success")
                print(reply)
        else:
                print("Delete Failure")

if __name__ == "__main__":
        loop = asyncio.get_event_loop()

        loop.run_until_complete(insert_one())
        loop.run_until_complete(find_one())
        loop.run_until_complete(update_one())
        loop.run_until_complete(delete_one())

Conclusion

After this, you'll be able to manipulate data from an external data service instead of relying on locally data management. Also with this knowledge, you would be able to log data from various sensors and offload it from the Raspberry Pi Zero 2W to MongoDB.