Skip to main content

Lucene Solr Backup Script

#!/bin/bash
solr_home="/opt/jboss/solr/";
solr_backup_folder="/solr_backup/";
today="$(date +%d%m%Y)";
todays_backup="backup.$(date +%Y%m%d)*"
index_name_tag1="corename=\"";
index_name_tag2="\"instanceDir";
instance_dir_tag1="instanceDir=\"";
instance_dir_tag2="\"/>";
logfile="/tmp/solr/solr_backup.log";
overall_result="ok";

rm -f ${logfile};

echo ${today}" solr backup logs \n" >> ${logfile};
echo "solr_home = "${solr_home}" \n" >> ${logfile};
echo "solr_backup_folder = "${solr_backup_folder}" \n" >> ${logfile};

cd ${solr_home};

cat ${solr_home}"solr.xml" | while read line
do
 line=$(echo $line|sed 's/ //g');
 index_name_prefix="${line%${index_name_tag1}*}";
 index_name_suffix="${line#*${index_name_tag2}}";
 index_name_noprefix="${line#${index_name_prefix}${index_name_tag1}}";
 instance_dir_prefix="${line%${instance_dir_tag1}*}";
 instance_dir_suffix="${line#*${instance_dir_tag2}}";
 instance_dir_noprefix="${line#${instance_dir_prefix}${instance_dir_tag1}}";

 if [ ! "$index_name_noprefix" == "$line" ]
   then
   index_name="${index_name_noprefix%${index_name_tag2}${index_name_suffix}}"
   instance_dir="${instance_dir_noprefix%${instance_dir_tag2}${instance_dir_suffix}}"
   echo "----- index name "${index_name}" \n"  >> ${logfile}
   echo "----- instance directory "${instance_dir}" \n" >> ${logfile}

   echo "removing old snapshots \n" >> ${logfile}
   cd ${instance_dir}"data"
   old_snapshots=`ls -d backup.2*`
   echo $old_snapshots" \n" >> ${logfile}
   rm -rf ${instance_dir}data/backup.2*

   echo "taking snapshot \n" >> ${logfile}
   cd ${solr_home}
   sh bin/backup -d ${index_name}/data
   cd ${instance_dir}"data/"
   solr_snapshot=`ls -td ${todays_backup}  | awk 'NR<2'`
   echo $solr_snapshot" \n" >> ${logfile}
   echo "copying snaphot to backup folder \n" >> ${logfile}
   cp -fr ${instance_dir}data/$solr_snapshot ${solr_backup_folder}${index_name}/

   cd ${solr_backup_folder}$index_name
   echo "controlling is backup copied \n" >> ${logfile}
   if [ ! -d "${solr_backup_folder}$index_name/$solr_snapshot" ] || [ "$solr_snapshot" == "" ]
     then
     echo $solr_snapshot" does not exist in "${solr_backup_folder}$index_name" \n" >> ${logfile}
     overall_result="no";
     else
     echo $solr_snapshot" exists in "${solr_backup_folder}$index_name" \n" >> ${logfile}
     echo "removing old backups \n" >> ${logfile}
     old_backups=`ls -td backup.* | awk 'NR>2'`
     echo $old_backups" \n" >>${logfile}
     ls -td backup.* | awk 'NR>2' | xargs rm -rf
   fi
 fi
done

mailtext=$(cat ${logfile})
if [ "$overall_result" == "ok" ]
  then
  echo -e $mailtext | mail -s "solr backups overall successful" solr_report@report.test.com
  else
  echo -e $mailtext | mail -s "solr backups overall failed !!!" solr_report@report.test.com
fi

Comments

  1. Hi,

    I'm sysadmin and new in Solr stuff..
    I've to backup solr collections that I've installed, and for that I want to inspire me of your script to make mine.

    But I don't have any file in solr home bin folder, so I don't have bin/backup command.
    I would like to know if this an another script of you that you place there, or if this is a solr command that I don't have ? Or something somewhere to copy at this place?

    I'm using Solr 4.3.

    I'm french so I hope my english can be understood :)

    In any case, just to say thank you for sharing your script !

    Best Regards

    ReplyDelete
    Replies
    1. Hi,
      When i wrote this script solrcloud was not introduced. It is normal not to see backup script in your bin folder. These Scripts were superseded by the ReplicationHandler Java implementation of index replication that works over HTTP and was introduced in Solr4, and are no longer actively maintained. So you should use replication option for backup purposes. Please look at this link: http://wiki.apache.org/solr/SolrReplication

      Delete

Post a Comment

Popular posts from this blog

Sending Jboss Server Logs to Logstash Using Filebeat with Multiline Support

In addition to sending system logs to logstash, it is possible to add a prospector section to the filebeat.yml for jboss server logs. Sometimes jboss server.log has single events made up from several lines of messages. In such cases Filebeat should be configured for a multiline prospector. Filebeat takes lines do not start with a date pattern (look at pattern in the multiline section "^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}" and negate section is set to true ) and combines them with the previous line that starts with a date pattern. server.log file excerpt where DatePattern: yyyy-MM-dd-HH and ConversionPattern: %d %-5p [%c] %m%n Logstash filter:

Creating Multiple VLANs over Bonding Interfaces with Proper Routing on a Centos Linux Host

In this post, I am going to explain configuring multiple VLANs on a bond interface. First and foremost, I would like to describe the environment and give details of the infrastructure. The server has 4 Ethernet links to a layer 3 switch with names: enp3s0f0, enp3s0f1, enp4s0f0, enp4s0f1 There are two bond interfaces both configured as active-backup bond0, bond1 enp4s0f0 and enp4s0f1 interfaces are bonded as bond0. Bond0 is for making ssh connections and management only so corresponding switch ports are not configured in trunk mode. enp3s0f0 and enp3s0f1 interfaces are bonded as bond1. Bond1 is for data and corresponding switch ports are configured in trunk mode. Bond0 is the default gateway for the server and has IP address 10.1.10.11 Bond1 has three subinterfaces with VLAN 4, 36, 41. IP addresses are 10.1.3.11, 10.1.35.11, 10.1.40.11 respectively. Proper communication with other servers on the network we should use routing tables. There are three