RabbitMQ integration with Apache Solr in Java

18 Sep 2023

Apache Solr is a popular open-source search platform that is widely used for building powerful search applications. RabbitMQ, on the other hand, is a robust message broker that enables seamless communication between different components of a system.

Integrating RabbitMQ with Apache Solr can be a great way to efficiently index and search large amounts of data. In this blog post, we will explore how to achieve this integration using Java.

Prerequisites

To follow along with this tutorial, you will need:

Apache Solr installed and running on your local machine or a remote server.
RabbitMQ installed and running on your local machine or a remote server.
Basic knowledge of Java programming language and RabbitMQ concepts.

Setup

Start by adding the RabbitMQ Java client dependency to your project. You can add it to your Maven pom.xml file or Gradle dependencies.

<dependency>
    <groupId>com.rabbitmq</groupId>
    <artifactId>amqp-client</artifactId>
    <version>5.12.0</version>
</dependency>

Next, you need to create a connection to RabbitMQ. Here’s an example code snippet to establish a connection and create a channel:

import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

public class RabbitMQConnection {
    private static final String RABBITMQ_HOST = "localhost";
    private static final int RABBITMQ_PORT = 5672;
    private static final String RABBITMQ_USERNAME = "guest";
    private static final String RABBITMQ_PASSWORD = "guest";

    public static Connection getConnection() throws IOException, TimeoutException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost(RABBITMQ_HOST);
        factory.setPort(RABBITMQ_PORT);
        factory.setUsername(RABBITMQ_USERNAME);
        factory.setPassword(RABBITMQ_PASSWORD);

        return factory.newConnection();
    }
}

Once you have a connection to RabbitMQ, you can start consuming messages from a queue and then index them in Apache Solr. Here’s an example code for consuming messages and indexing them in Solr using the SolrJ library:

import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.impl.HttpSolrClient;
import org.apache.solr.common.SolrInputDocument;

public class RabbitMqConsumer {
    private static final String RABBITMQ_QUEUE_NAME = "my_queue";
    private static final String SOLR_URL = "http://localhost:8983/solr/";

    public static void consumeAndIndexMessages() throws IOException, TimeoutException, InterruptedException, SolrServerException {
        Connection connection = RabbitMQConnection.getConnection();
        Channel channel = connection.createChannel();
        channel.queueDeclare(RABBITMQ_QUEUE_NAME, false, false, false, null);

        SolrClient solrClient = new HttpSolrClient.Builder(SOLR_URL).build();

        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
                String message = new String(body, "UTF-8");
                System.out.println("Received message: " + message);

                // Create Solr document
                SolrInputDocument doc = new SolrInputDocument();
                doc.addField("content", message);

                // Add document to Solr
                solrClient.add(doc);
                solrClient.commit();

                System.out.println("Indexed message in Solr: " + message);
            }
        };

        channel.basicConsume(RABBITMQ_QUEUE_NAME, true, consumer);
    }
}

Conclusion

By integrating RabbitMQ with Apache Solr in Java, you can achieve efficient and scalable search capabilities for your applications. This tutorial provided a basic example to get you started, but there are many more advanced features and optimizations you can explore.

Remember to start RabbitMQ and Apache Solr before running the Java application. Don’t forget to configure your queues, exchange, and Solr core according to your needs.

#hashtags: #RabbitMQ #ApacheSolr