Like Us On Facebook

Follow Us On Twitter

Drop Down Menu

Friday 15 February 2019

A Clustering Guide to CrateDB and ElasticSearch

Hi friends,

Recently I ran CrateDB in multi-node environment as a trial and in this post I'll share my findings as to what is the basic setup and configuration that is required to have high availability setup in CrateDB. I am pretty sure that similar settings will be useful for ElasticSearch as well.

I used crateDB version 3.2.2 and Java 8 for this POC.

So there is some bootstrap checks before the server runs successfully. According to the documentation those are some mandatory checks that are mandatory and needed to get the cluster running smoothly in multi-node environment. 

The first problem I faced was related to increasing ulimit in Ubuntu, which I have explained in this post - Increase Open Files/File Descriptors/Ulimit in Ubuntu | CrateDB ElasticSearch.

Once my CrateDB ran successfully, I began to mess around with basic settings and configurations. Remember, many of the things can be customised, and I'll share a few read-worthy links if you truly want to customise all settings around your cluster, towards the end of this post.

So here's the final configuration that worked for me:


Increase Open Files/File Descriptors/Ulimit in Ubuntu | CrateDB ElasticSearch

Hi friends,

This week I faced a rather peculiar issue while trying to setup clustering in CrateDB, which is very much similar to the clustering done in ElasticSearch. As soon as I did the basic configuration which you can find here, starting the process showed the following kinds of exceptions:



Thursday 14 February 2019

Switch Between Multiple Java Versions on Linux

Hi friends,

In my previous blog post - Install Java 11 and Java 8 on Linux, I shared the way to install Java 11/Java 8 on your Linux machine. Now many of the current applications do not support Java 11 yet, so you must need a way to quickly switch between different installations of Java on your Linux machine. Without further ado, let's get started.


Way 1:

If you actually followed my way of installing Java 8/ Java 11 on your machine, and did set Java 8 / Java 11 default by using either of these commands:

sudo apt install oracle-java8-set-default

sudo apt install oracle-java11-set-default

Simply install the other command and it will automatically set that version as default in your system.

Now if you have both the above packages installed and now you want to set either of them as default, simply purge the one you wish to set as default and reinstall it.

So if you want to set Java 8 as default, run these commands in your terminal in order:

sudo apt purge oracle-java8-set-default

sudo apt install oracle-java8-set-default


If you want to set Java 11 as default, run these commands in your terminal in order:

sudo apt purge oracle-java11-set-default

sudo apt install oracle-java11-set-default



Way 2:




Sunday 10 February 2019

Install Java 11 and Java 8 on Linux

Hello friends,

As many of you must be knowing that Oracle officially ended the support for Java 8 in January this year. What this means for you? You get to try your hands on the latest JDK, which is 11.0.2 as of now. I will cover practical daily-code uses of Java 11 in a separate post soon. But till that time, let's discuss how to install Java 11 on your Linux distro. 

Now since many of your applications might still be using Java 8 and switching to Java 11 on production might not sound the best idea to your manager. This leaves you to find a way to install multiple Java versions on your machine and be able to switch between them in real-time.



In this post I'll share the steps to install Java 8 and Java 11 on your Linux machine and you can find how to switch between these two installations in this post -> How to switch between Java versions on Linux. 



Monday 14 January 2019

Log4j SMTP Appender Using System Environment Variables

Hey friends!

Earlier last week, I was stuck for a short while in a what should've been an easy task, but took my more than expected working hours to completely solve that: SMTP appender for Log4j.

So as we all know, SMTP or Simple Mail Transfer Protocol is used to send emails from your java program, generally when a particular level of exception occurs in your code. Log4j is a well known Java logging utility, but this particular appender wasn't discussed as about much as I expected earlier. 


LOG4J SMTP APPENDER

Earlier I was using log4j2.yml file for logging but due to some versioning reasons, it wasn't picking up the Log4j keywords properly from the version of Log4j that I added. So eventually I had to shift to using the XML version.



Thursday 10 January 2019

Fixed — ElasticSearch CrateDB Logstash — ClusterBlockException Forbidden Index Read-Only

Hey friends,

Ever since I started working with ElasticSearch, CrateDB and Logstash, I have faced plenty of issues and exceptions that maybe had a clear reason and fix but were not documented well anywhere. I had to research through tons and tons of documentation and StackOverFlow pages to atleast get some hints as to what might be causing the issue and how to fix it. 


It was then I decided that I will document such problems for easy access and quick solution for others. One such exception that I got multiple times was:



09:46:07.741 [main] ERROR
 io.crate.shade.org.postgresql.util.PSQLException: ERROR: ClusterBlockException: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];
    at io.crate.shade.org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2458)
    at io.crate.shade.org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2158)
    at io.crate.shade.org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:291)


This was due to:
[WARN ][o.e.c.r.a.DiskThresholdMonitor] [Piz Roseg] high disk watermark [90%] exceeded on [Piz Roseg][/crate/
[WARN ][o.e.c.r.a.DiskThresholdMonitor] [Piz Roseg] high disk watermark [90%] exceeded on [Piz Roseg][/crate/
[WARN ][o.e.c.r.a.DiskThresholdMonitor] [Piz Roseg] rerouting shards: [high disk watermark exceeded on one or more nodes]
[WARN ][o.e.c.r.a.DiskThresholdMonitor] [Piz Roseg] high disk watermark [90%] exceeded on [Piz Roseg][/crate/
[WARN ][o.e.c.r.a.DiskThresholdMonitor] [Piz Roseg] high disk watermark [90%] exceeded on [Piz Roseg][/crate/
[INFO ][o.e.c.r.a.DiskThresholdMonitor] [Piz Roseg] rerouting shards: [high disk watermark exceeded on one or more nodes]
[WARN ][o.e.c.r.a.DiskThresholdMonitor] [Piz Roseg] high disk watermark [90%] exceeded on [Piz Roseg][/crate/
[WARN ][o.e.c.r.a.DiskThresholdMonitor] [Piz Roseg] high disk watermark [90%] exceeded on [Piz Roseg][/crate/
[INFO ][o.e.c.r.a.DiskThresholdMonitor] [Piz Roseg] rerouting shards: [high disk watermark exceeded on one or more nodes]

The Cause:

This particular exception is caused by a very simple problem - Disk Space. You are definitely running out of disk space on the disk where ElasticSearch/Logstash/CrateDB is installed. 



Wednesday 9 January 2019

Add/Edit Startup Applications in Ubuntu Linux

Hey friends,

I recently switched over to Ubuntu and I must say it feels a better and crunchier experience as compared to my alternate previous OS, the Windows. That said, switching over to Ubuntu brought its own set of problems and Ububtu 18.10 (Bionic Beaver) is not as stable as 17.04 was for me. 

So today I am going to share a very easy way to edit the startup applications on your Ubuntu machine. This approach should work for previous versions as well.

Now just like Windows, there are some startup applications in Linux which can be added/deleted as per your own convenience. Without further ado, let me show you how:

  • The most easy way is to first press Alt + F2 keys together.

  • Then write gnome-session-properties and press Enter. You will be presented with the following screen.