Elasticsearch could not be restarted

subratan 2018-02-14 10:39:31 UTC #22

Hi Juan,
Here is the output

wizard4property@elasticsearch-cutecars-vm:~$ sudo mv /opt/bitnami/elasticsearch/plugins/master.zip ~
wizard4property@elasticsearch-cutecars-vm:~$ sudo /opt/bitnami/ctlscript.sh restart elasticsearch
Unmonitored elasticsearch
/opt/bitnami/elasticsearch/scripts/ctl.sh : elasticsearch not running

Thanks,
Subrata Nandi

jariza 2018-02-14 10:43:24 UTC #23

Hi @subratan

What's the error you obtain on the ElasticSearch log now?

Best Regards,

Juan Ariza

subratan 2018-02-14 10:52:07 UTC #24

Hello Juan,

Here is the login content:

[2018-02-14T10:38:01,514][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: property [elasticsearch.version] is missing for plugin [head]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.1.1.jar:6.1.1]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.1.1.jar:6.1.1]
Caused by: java.lang.IllegalArgumentException: property [elasticsearch.version] is missing for plugin [head]
        at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:144) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Spawner.spawnNativePluginControllers(Spawner.java:80) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:166) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:322) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.1.1.jar:6.1.1]
        ... 6 more
[2018-02-14T10:40:45,870][ERROR][o.e.b.Bootstrap          ] Exception
java.lang.IllegalArgumentException: property [elasticsearch.version] is missing for plugin [head]
        at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:144) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Spawner.spawnNativePluginControllers(Spawner.java:80) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:166) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:322) [elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.1.1.jar:6.1.1]
        at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.1.1.jar:6.1.1]
[2018-02-14T10:40:45,882][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: property [elasticsearch.version] is missing for plugin [head]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.1.1.jar:6.1.1]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.1.1.jar:6.1.1]
Caused by: java.lang.IllegalArgumentException: property [elasticsearch.version] is missing for plugin [head]
        at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:144) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Spawner.spawnNativePluginControllers(Spawner.java:80) ~[elasticsearch-6.1.1.jar:6.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:166) ~[elasticsearch-6.1.1.jar:6.1.1]

Thanks,
Subrata Nandi

subratan 2018-02-15 06:12:19 UTC #25

Hello Juan,

Is this log file helping to sort out the issue?

Regards,
Subrata Nandi

jariza 2018-02-15 10:27:27 UTC #26

Hello @subratan

It seems there's a property missing on the plugin you have installed at /opt/bitnami/elasticsearch/plugins/elasticsearch-head-master. See the error below:

java.lang.IllegalArgumentException: property [elasticsearch.version] is missing for plugin [head]

It seems the plugin-descriptor.properties file includen on the plugin is missing some properties such as [elasticsearch.version].

Are you using that plugin for anything? If not, just remove it.

Best Regards,

Juan Ariza

subratan 2018-02-15 11:18:24 UTC #27

Hello Juan,

I am not using any plugins, only I need to access elastic search using Angular js that was successful to connect. Only the issue was CORS issue that was also resolved but all of a sudden when I tried to open elastic from browser the message given "Your application is not available
Your application may not be available for many reasons: BLA BLA BLA". However I renamed the file as plugin-descriptor.properties_prev under
vm:~/stack/elasticsearch/plugins/elasticsearch-head-master$ and ran
sudo /opt/bitnami/ctlscript.sh restart elasticsearch from ........vm:~$ and giving error as
Unmonitored elasticsearch
/opt/bitnami/elasticsearch/scripts/ctl.sh : elasticsearch not running

Thanks,
Subrata Nandi

subratan 2018-02-16 06:36:26 UTC #28

Hello Juan,

I removed plugin-descriptor.properties plugin but still elastic not running.

Regards,
Subrata Nandi

jariza 2018-02-16 09:16:53 UTC #29

Hello @subratan

Could you try the following?

Create a backup directory.

sudo mkdir /home/bitnami/backup/

Move all the plugins to that backup directory.

sudo mv /opt/bitnami/elasticsearch/plugins/elasticsearch-head-master  /home/bitnami/backup/

Restart Elasticsearch.

sudo /opt/bitnami/ctlscript.sh restart elasticsearch

Paste the log file in case this last workaround doesn't work.

Best Regards,

Juan Ariza

subratan 2018-02-16 10:05:29 UTC #30

Hello Juan,
I followed as per given your instruction but sorry fails!
-vm:~$ sudo mv /opt/bitnami/elasticsearch/plugins/elasticsearch-head-master /home/bitnami/backup/
-vm:~$ sudo /opt/bitnami/ctlscript.sh restart elasticsearch
Unmonitored elasticsearch
/opt/bitnami/elasticsearch/scripts/ctl.sh : elasticsearch not running

LOG FILE CONTENT:

[2018-02-16T10:00:04,227][INFO ][o.e.n.Node               ] [] initializing ...
[2018-02-16T10:00:04,673][INFO ][o.e.e.NodeEnvironment    ] [ZWzBmnm] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [6.3gb], net total_space [9.7gb], ty$
[2018-02-16T10:00:04,679][INFO ][o.e.e.NodeEnvironment    ] [ZWzBmnm] heap size [247.6mb], compressed ordinary object pointers [true]
[2018-02-16T10:00:04,896][INFO ][o.e.n.Node               ] node name [ZWzBmnm] derived from node ID [ZWzBmnmhRyuQwYHl2pFMvA]; set [node.name] to override
[2018-02-16T10:00:04,897][INFO ][o.e.n.Node               ] version[6.1.1], pid[24911], build[bd92e7f/2017-12-17T20:23:25.338Z], OS[Linux/3.16.0-4-amd64/amd64], JVM[Ora$
[2018-02-16T10:00:04,897][INFO ][o.e.n.Node               ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSIn$
[2018-02-16T10:00:08,215][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [aggs-matrix-stats]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [analysis-common]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [ingest-common]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [lang-expression]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [lang-mustache]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [lang-painless]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [mapper-extras]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [parent-join]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [percolator]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [reindex]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [repository-url]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [transport-netty4]
[2018-02-16T10:00:08,216][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] loaded module [tribe]
[2018-02-16T10:00:08,217][INFO ][o.e.p.PluginsService     ] [ZWzBmnm] no plugins loaded
[2018-02-16T10:00:14,681][INFO ][o.e.d.DiscoveryModule    ] [ZWzBmnm] using discovery type [zen]
[2018-02-16T10:00:16,515][INFO ][o.e.n.Node               ] initialized
[2018-02-16T10:00:16,517][INFO ][o.e.n.Node               ] [ZWzBmnm] starting ...
[2018-02-16T10:00:17,010][INFO ][o.e.t.TransportService   ] [ZWzBmnm] publish_address {35.192.211.136:9300}, bound_addresses {[::]:9300}
[2018-02-16T10:00:17,065][INFO ][o.e.b.BootstrapChecks    ] [ZWzBmnm] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2018-02-16T10:00:20,257][INFO ][o.e.c.s.MasterService    ] [ZWzBmnm] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {ZWzBmnm}{ZWzBmnmhRyuQwYHl2pFMv$
[2018-02-16T10:00:20,307][INFO ][o.e.c.s.ClusterApplierService] [ZWzBmnm] new_master {ZWzBmnm}{ZWzBmnmhRyuQwYHl2pFMvA}{OVfnh7Y2T7WS_WC3DH1Wtg}{35.192.211.136}{35.192.21$
[2018-02-16T10:00:20,974][INFO ][o.e.h.n.Netty4HttpServerTransport] [ZWzBmnm] publish_address {35.192.211.136:9200}, bound_addresses {[::]:9200}
[2018-02-16T10:00:20,974][INFO ][o.e.n.Node               ] [ZWzBmnm] started
[2018-02-16T10:00:20,973][INFO ][o.e.m.j.JvmGcMonitorService] [ZWzBmnm] [gc][4] overhead, spent [580ms] collecting in the last [1.2s]
[2018-02-16T10:00:21,833][INFO ][o.e.g.GatewayService     ] [ZWzBmnm] recovered [3] indices into cluster_state
[2018-02-16T10:00:23,392][INFO ][o.e.c.r.a.AllocationService] [ZWzBmnm] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[cute_car][1]] ..$

Thanks,
Subrata Nandi

jariza 2018-02-16 10:33:20 UTC #31

Hello @subratan

Hm... It seems it was initialised...

[2018-02-16T10:00:04,227][INFO ][o.e.n.Node               ] [] initializing ...
...
[2018-02-16T10:00:16,515][INFO ][o.e.n.Node               ] initialized
[2018-02-16T10:00:16,517][INFO ][o.e.n.Node               ] [ZWzBmnm] starting ...
...
[2018-02-16T10:00:20,974][INFO ][o.e.n.Node               ] [ZWzBmnm] started

Could you run the command below and share the output?

ps aux | grep "elasticsearch"

Best Regards,

Juan Ariza

subratan 2018-02-16 10:37:49 UTC #32

Hello Juan,

After running the command , here is the output:

elastic+ 24911  1.0 63.2 2333076 385004 ?      Sl   09:59   0:22 /opt/bitnami/java/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -XX:+HeapDumpOnOutOfMemoryError -XX:MaxMetaspaceSize=256M -Xms256M -Xmx256M -Des.path.home=/opt/bitnami/elasticsearch -Des.path.conf=/opt/bitnami/elasticsearch/config -cp /opt/bitnami/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -d -p/opt/bitnami/elasticsearch/tmp/elasticsearch.pid
wizard4+ 25347  0.0  0.3  12728  2172 pts/0    S+   10:36   0:00 grep elasticsearch

Thanks,
Subrata Nandi

jariza 2018-02-16 10:39:30 UTC #33

Hi @subratan

There you go!! Elasticsearch is running!!! Could you try to connect to it? The message could be an error on the script to detect if it's running or not:

/opt/bitnami/elasticsearch/scripts/ctl.sh : elasticsearch not running

However, it seems it's running.

Best Regards,

Juan Ariza

subratan 2018-02-16 10:45:50 UTC #34

Hello Juan,

Thanks a lot !!!!!!!!!!!!!!!!!!!!!!!!! It is running. If any problem I will let you know.

Again thanks,

Subrata Nandi

jariza 2018-02-19 10:31:34 UTC #35

Hi @subratan

You're welcome. We should investigate the reason why the ctlscript.sh script is unable to control the service. Could you run?

cat /opt/bitnami/elasticsearch/tmp/elasticsearch.pid

Maybe the PID information is wrong and that's all.

Best Regards,

Juan Ariza

subratan 2018-02-21 07:17:24 UTC #36

Hello Juan,

Sorry for the delay answering the issue. Yes I ran the command as you mentioned and output is nothing.

vm:~$ cat /opt/bitnami/elasticsearch/tmp/elasticsearch.pid
vm:~$

Thanks,
Subrata Nandi

jariza 2018-02-21 09:43:53 UTC #37

Hi @subratan

I suspected that.. Could you please do the following actions?

sudo /opt/bitnami/ctlscript.sh stop elasticsearch
sudo rm /opt/bitnami/elasticsearch/tmp/elasticsearch.pid
sudo /opt/bitnami/ctlscript.sh start elasticsearch

Let's see if now you don't get the message:

/opt/bitnami/elasticsearch/scripts/ctl.sh : elasticsearch not running

Best Regards,

Juan Ariza

subratan 2018-02-21 09:58:56 UTC #38

Hi Juan,

Here is the command run and output:

vm:~$ sudo /opt/bitnami/ctlscript.sh stop elasticsearch
Unmonitored elasticsearch
/opt/bitnami/elasticsearch/scripts/ctl.sh : elasticsearch stopped
vm:~$ sudo rm /opt/bitnami/elasticsearch/tmp/elasticsearch.pid
rm: cannot remove ‘/opt/bitnami/elasticsearch/tmp/elasticsearch.pid’: No such file or directory
vm:~$ sudo /opt/bitnami/ctlscript.sh start elasticsearch
/opt/bitnami/elasticsearch/scripts/ctl.sh : elasticsearch started
Monitored elasticsearch
vm:~$

Thanks,
Subrata Nandi

jariza 2018-02-21 10:02:45 UTC #39

Hi @subratan

That's perfect!! As you can see now you obtained a successful result when you started Elasticsearch:

$ sudo /opt/bitnami/ctlscript.sh start elasticsearch
/opt/bitnami/elasticsearch/scripts/ctl.sh : elasticsearch started
Monitored elasticsearch

Now you can run again the command below and you'll obtain the PID of your Elasticsearch service.

sudo cat /opt/bitnami/elasticsearch/tmp/elasticsearch.pid

I think it's everything fine now. Do you need any more help from my side?

Best Regards,

Juan Ariza

subratan 2018-02-21 10:33:04 UTC #40

Hi Juan,

Here is the output , it is seen extreme left 30234, is it the pid?

xxxxxxxx@elasticsearch-xxxx-vm:~$ sudo cat /opt/bitnami/elasticsearch/tmp/elasticsearch.pid
30234xxxxxxxxxx@elasticsearch-vm:~$

Regards,
Subrata Nandi

crhernandez 2018-02-21 11:54:11 UTC #41

Hi,

Yes, this number is the Elasticsearch process pid.

If you run

sudo /opt/bitnami/ctlscript.sh status

you can check that everything is working fine.

Best regards,
Carlos R. Hernández

Bitnami provides free all-in-one installers, virtual machines and cloud images for popular open source applications. Get them now!