Understanding Avro file with example
Today in this article, we will learn about Avro file format with examples.
Avro is an open-source schema specification for data serialization that provides serialization and data exchange services for Apache Hadoop.
Today, we will cover the below feature of the Avro file,
What is Avro
Avro is a language-agnostic format that can be used for any language that facilitates the exchange of data between programs. Today in this article we will see Avro file with an example.
Benefits of Avro
- Serialize/Deserialize data into files or into messages.
- The data storage is compact and efficient.
- Rich data structure.
- The Data is stored in a binary format making it compact and efficient.
- Data definition is stored in JSON format making it easy to read and interpret.
- It can be used with a scripting language.
- Can be used for Remote procedure calls (RPC).
- A key feature of Avro is backward compatibility with support for data schemas older and new ones.
Avro schema lets you define all the possible Types including nullable if any for available properties.
We already learned, how to convert JSON into Avero schema and vice versa – Generate Avro Schema from JSON
Below are a few examples of Avro schema which you can refer to for understanding purposes.
Avro – Basic example 1
Below is a basic sample file of Avro schema.
{
"type": "record",
"name": "thecodebuzz_schema",
"namespace": "thecodebuzz.avro",
"fields": [
{
"name": "username",
"type": "string",
"doc": "Name of the user account on Thecodebuzz.com"
},
{
"name": "email",
"type": "string",
"doc": "The email of the user logging message on the blog"
},
{
"name": "timestamp",
"type": "long",
"doc": "time in seconds"
}
],
"doc:": "A basic schema for storing thecodebuzz blogs messages"
}Avro Example 2 – with Array and null values
Below is a basic sample file of Avro schema with array and NULL value representation
{
"type": "record",
"name": "AvrosampleNetCore.AccountDetails",
"fields": [
{
"name": "AccountId",
"type": "int"
},
{
"name": "AccountName",
"type": "string"
},
{
"name": "Accounts",
"type": [
"null",
{
"type": "array",
"items": {
"type": "record",
"name": "AvrosampleNetCore.SubAccounts",
"fields": [
{
"name": "AccountId",
"type": "int"
},
{
"name": "AccountType",
"type": [ "null", "string" ]
}
]
}
}
]
}
]
}Avro File Serialization and deserialazation
If you are interested in learning Avro file Serialization and deserialization,
References :
Do you have any comments or ideas or any better suggestions to share?
Please sound off your comments below.
Happy Coding !!
Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published(2024) best practices and guidelines for software design and development.
Hi there, I was trying to read the content “Avro File Serialization and deserialzation” nut seems like it contains the link of another article “Generate Avro Schema from JSON”. Hope you will fix this!
Hi Rafiul – You can refer to the same article for serializing a type into Avro schema. I have not yet covered the deserialization part in the article.