Database: ERROR (spawn error) after re-installation

Hi,
I had to reinstall cryosparc after a complete rebuild of our machine.
Before, I copied the whole database to a separate disk and copied back the folder now.
Installation of cryosparc goes smoothly, however upon startup the spawn error appears.
This is only true for the old database, if I create a new it works.
Already deleted all lock files in the database path without success.
How should I proceed?

Best,
tarek

Hi @tarek,

Can you provide the output of cryosparcm log database
This will show the output of the database as it tried to start up. Most likely this is a permissions issue.

Can you also check if there are any “zombie” cryosparc processes that are running? After cryosparcm stop you should see no cryosparc processes being run in htop .

Hi Ali,
no zombie processes left, below you can find the logs.
Thanks for your support!

> 2020-02-14T11:42:29.029+0100 I CONTROL  [initandlisten] MongoDB starting : pid=8194 port=39001 dbpath=/srv/public/cryosparc2/cryosparc2_database 64-bit host=gpu031
2020-02-14T11:42:29.029+0100 I CONTROL  [initandlisten] db version v3.4.10-4-g67ee356c6b
2020-02-14T11:42:29.029+0100 I CONTROL  [initandlisten] git version: 67ee356c6be377cda547d16423daef5beb4e8377
2020-02-14T11:42:29.029+0100 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2p  14 Aug 2018
2020-02-14T11:42:29.029+0100 I CONTROL  [initandlisten] allocator: tcmalloc
2020-02-14T11:42:29.029+0100 I CONTROL  [initandlisten] modules: none
2020-02-14T11:42:29.029+0100 I CONTROL  [initandlisten] build environment:
2020-02-14T11:42:29.029+0100 I CONTROL  [initandlisten]     distarch: x86_64
2020-02-14T11:42:29.029+0100 I CONTROL  [initandlisten]     target_arch: x86_64
2020-02-14T11:42:29.029+0100 I CONTROL  [initandlisten] options: { net: { port: 39001 }, replication: { oplogSizeMB: 64, replSet: "meteor" }, storage: { dbPath: "/srv/public/cryosparc2/cryosparc2_database", journal: { enabled: false }, wiredTiger: { engineConfig: { cacheSizeGB: 4.0 } } } }
2020-02-14T11:42:29.057+0100 I -        [initandlisten] Detected data files in /srv/public/cryosparc2/cryosparc2_database created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-02-14T11:42:29.057+0100 I STORAGE  [initandlisten]
2020-02-14T11:42:29.057+0100 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2020-02-14T11:42:29.057+0100 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-02-14T11:42:29.057+0100 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=4096M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),,log=(enabled=false),
2020-02-14T11:42:29.078+0100 E STORAGE  [initandlisten] WiredTiger error (-31802) [1581676949:78473][8194:0x7f172f2edd40], file:sizeStorer.wt, WT_SESSION.open_cursor: unable to read root page from file:sizeStorer.wt: WT_ERROR: non-specific WiredTiger error
2020-02-14T11:42:29.078+0100 I -        [initandlisten] Invariant failure: ret resulted in status UnknownError: -31802: WT_ERROR: non-specific WiredTiger error at src/mongo/db/storage/wiredtiger/wiredtiger_size_storer.cpp 69
2020-02-14T11:42:29.078+0100 I -        [initandlisten]

***aborting after invariant() failure


2020-02-14T11:42:29.082+0100 F -        [initandlisten] Got signal: 6 (Aborted).

 0x559a8ac7bd6b 0x559a8ac7afe6 0x559a8ac7b4c3 0x7f172f4ca730 0x7f172f32e7bb 0x7f172f319535 0x559a89e9a739 0x559a8a96d8dd 0x559a8a957758 0x559a8a94f9ca 0x559a8a83550a 0x559a89ea6bab 0x7f172f31b09b 0x559a89f0acff
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"559A8985B000","o":"1420D6B","s":"_ZN5mongo15printStackTraceERSo"},{"b":"559A8985B000","o":"141FFE6"},{"b":"559A8985B000","o":"14204C3"},{"b":"7F172F4B8000","o":"12730"},{"b":"7F172F2F7000","o":"377BB","s":"gsignal"},{"b":"7F172F2F7000","o":"22535","s":"abort"},{"b":"559A8985B000","o":"63F739","s":"_ZN5mongo25fassertFailedWithLocationEiPKcj"},{"b":"559A8985B000","o":"11128DD","s":"_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"},{"b":"559A8985B000","o":"10FC758","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb"},{"b":"559A8985B000","o":"10F49CA"},{"b":"559A8985B000","o":"FDA50A","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"559A8985B000","o":"64BBAB","s":"main"},{"b":"7F172F2F7000","o":"2409B","s":"__libc_start_main"},{"b":"559A8985B000","o":"6AFCFF"}],"processInfo":{ "mongodbVersion" : "3.4.10-4-g67ee356c6b", "gitVersion" : "67ee356c6be377cda547d16423daef5beb4e8377", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.19.0-6-amd64", "version" : "#1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11)", "machine" : "x86_64" }, "somap" : [ { "b" : "559A8985B000", "elfType" : 3, "buildId" : "A14732EAAA95508462633A5030B501E08B1B106F" }, { "b" : "7FFED86BE000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "50CD71D25893F0071A09BC55A2197F59A787ADCB" }, { "b" : "7F172FEDD000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0", "elfType" : 3 }, { "b" : "7F172FC67000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libssl.so.1.0.0", "elfType" : 3 }, { "b" : "7F172F826000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libcrypto.so.1.0.0", "elfType" : 3 }, { "b" : "7F172F7EB000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "5DCF98AD684962BE494AF28A1051793FD39E4EBC" }, { "b" : "7F172F7E6000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D3583C742DD47AAA860C5AE0C0C5BDBCD2D54F61" }, { "b" : "7F172F670000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libstdc++.so.6", "elfType" : 3 }, { "b" : "7F172F4ED000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "885DDA4B4A5CEA600E7B5B98C1AD86996C8D2299" }, { "b" : "7F172F4D9000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7F172F4B8000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "E91114987A0147BD050ADDBD591EB8994B29F4B3" }, { "b" : "7F172F2F7000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "18B9A9A8C523E5CFE5B5D946D605D09242F09798" }, { "b" : "7F172F2F2000", "path" : "/lib/x86_64-linux-gnu/libutil.so.1", "elfType" : 3, "buildId" : "F44B754DCF4119E5743708FED89461489880474D" }, { "b" : "7F17300BF000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "F25DFD7B95BE4BA386FD71080ACCAE8C0732B711" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x3B) [0x559a8ac7bd6b]
 mongod(+0x141FFE6) [0x559a8ac7afe6]
 mongod(+0x14204C3) [0x559a8ac7b4c3]
 libpthread.so.0(+0x12730) [0x7f172f4ca730]
 libc.so.6(gsignal+0x10B) [0x7f172f32e7bb]
 libc.so.6(abort+0x121) [0x7f172f319535]
 mongod(_ZN5mongo25fassertFailedWithLocationEiPKcj+0x0) [0x559a89e9a739]
 mongod(_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x1DD) [0x559a8a96d8dd]
 mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb+0x8E8) [0x559a8a957758]
 mongod(+0x10F49CA) [0x559a8a94f9ca]
 mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x69A) [0x559a8a83550a]
 mongod(main+0xE9B) [0x559a89ea6bab]
 libc.so.6(__libc_start_main+0xEB) [0x7f172f31b09b]
 mongod(+0x6AFCFF) [0x559a89f0acff]
-----  END BACKTRACE  -----
2020-02-14T11:42:30.129+0100 I CONTROL  [initandlisten] MongoDB starting : pid=8203 port=39001 dbpath=/srv/public/cryosparc2/cryosparc2_database 64-bit host=gpu031
2020-02-14T11:42:30.129+0100 I CONTROL  [initandlisten] db version v3.4.10-4-g67ee356c6b
2020-02-14T11:42:30.129+0100 I CONTROL  [initandlisten] git version: 67ee356c6be377cda547d16423daef5beb4e8377
2020-02-14T11:42:30.129+0100 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2p  14 Aug 2018
2020-02-14T11:42:30.129+0100 I CONTROL  [initandlisten] allocator: tcmalloc
2020-02-14T11:42:30.129+0100 I CONTROL  [initandlisten] modules: none
2020-02-14T11:42:30.129+0100 I CONTROL  [initandlisten] build environment:
2020-02-14T11:42:30.129+0100 I CONTROL  [initandlisten]     distarch: x86_64
2020-02-14T11:42:30.129+0100 I CONTROL  [initandlisten]     target_arch: x86_64
2020-02-14T11:42:30.129+0100 I CONTROL  [initandlisten] options: { net: { port: 39001 }, replication: { oplogSizeMB: 64, replSet: "meteor" }, storage: { dbPath: "/srv/public/cryosparc2/cryosparc2_database", journal: { enabled: false }, wiredTiger: { engineConfig: { cacheSizeGB: 4.0 } } } }
2020-02-14T11:42:30.157+0100 I -        [initandlisten] Detected data files in /srv/public/cryosparc2/cryosparc2_database created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-02-14T11:42:30.157+0100 I STORAGE  [initandlisten]
2020-02-14T11:42:30.157+0100 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2020-02-14T11:42:30.157+0100 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-02-14T11:42:30.157+0100 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=4096M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),,log=(enabled=false),
2020-02-14T11:42:30.177+0100 E STORAGE  [initandlisten] WiredTiger error (-31802) [1581676950:177799][8203:0x7fb2ab144d40], file:sizeStorer.wt, WT_SESSION.open_cursor: unable to read root page from file:sizeStorer.wt: WT_ERROR: non-specific WiredTiger error
2020-02-14T11:42:30.177+0100 I -        [initandlisten] Invariant failure: ret resulted in status UnknownError: -31802: WT_ERROR: non-specific WiredTiger error at src/mongo/db/storage/wiredtiger/wiredtiger_size_storer.cpp 69
2020-02-14T11:42:30.177+0100 I -        [initandlisten]

***aborting after invariant() failure


2020-02-14T11:42:30.181+0100 F -        [initandlisten] Got signal: 6 (Aborted).

 0x560528257d6b 0x560528256fe6 0x5605282574c3 0x7fb2ab321730 0x7fb2ab1857bb 0x7fb2ab170535 0x560527476739 0x560527f498dd 0x560527f33758 0x560527f2b9ca 0x560527e1150a 0x560527482bab 0x7fb2ab17209b 0x5605274e6cff
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"560526E37000","o":"1420D6B","s":"_ZN5mongo15printStackTraceERSo"},{"b":"560526E37000","o":"141FFE6"},{"b":"560526E37000","o":"14204C3"},{"b":"7FB2AB30F000","o":"12730"},{"b":"7FB2AB14E000","o":"377BB","s":"gsignal"},{"b":"7FB2AB14E000","o":"22535","s":"abort"},{"b":"560526E37000","o":"63F739","s":"_ZN5mongo25fassertFailedWithLocationEiPKcj"},{"b":"560526E37000","o":"11128DD","s":"_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"},{"b":"560526E37000","o":"10FC758","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb"},{"b":"560526E37000","o":"10F49CA"},{"b":"560526E37000","o":"FDA50A","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"560526E37000","o":"64BBAB","s":"main"},{"b":"7FB2AB14E000","o":"2409B","s":"__libc_start_main"},{"b":"560526E37000","o":"6AFCFF"}],"processInfo":{ "mongodbVersion" : "3.4.10-4-g67ee356c6b", "gitVersion" : "67ee356c6be377cda547d16423daef5beb4e8377", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.19.0-6-amd64", "version" : "#1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11)", "machine" : "x86_64" }, "somap" : [ { "b" : "560526E37000", "elfType" : 3, "buildId" : "A14732EAAA95508462633A5030B501E08B1B106F" }, { "b" : "7FFCFFD24000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "50CD71D25893F0071A09BC55A2197F59A787ADCB" }, { "b" : "7FB2ABD34000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0", "elfType" : 3 }, { "b" : "7FB2ABABE000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libssl.so.1.0.0", "elfType" : 3 }, { "b" : "7FB2AB67D000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libcrypto.so.1.0.0", "elfType" : 3 }, { "b" : "7FB2AB642000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "5DCF98AD684962BE494AF28A1051793FD39E4EBC" }, { "b" : "7FB2AB63D000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D3583C742DD47AAA860C5AE0C0C5BDBCD2D54F61" }, { "b" : "7FB2AB4C7000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libstdc++.so.6", "elfType" : 3 }, { "b" : "7FB2AB344000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "885DDA4B4A5CEA600E7B5B98C1AD86996C8D2299" }, { "b" : "7FB2AB330000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7FB2AB30F000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "E91114987A0147BD050ADDBD591EB8994B29F4B3" }, { "b" : "7FB2AB14E000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "18B9A9A8C523E5CFE5B5D946D605D09242F09798" }, { "b" : "7FB2AB149000", "path" : "/lib/x86_64-linux-gnu/libutil.so.1", "elfType" : 3, "buildId" : "F44B754DCF4119E5743708FED89461489880474D" }, { "b" : "7FB2ABF16000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "F25DFD7B95BE4BA386FD71080ACCAE8C0732B711" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x3B) [0x560528257d6b]
 mongod(+0x141FFE6) [0x560528256fe6]
 mongod(+0x14204C3) [0x5605282574c3]
 libpthread.so.0(+0x12730) [0x7fb2ab321730]
 libc.so.6(gsignal+0x10B) [0x7fb2ab1857bb]
 libc.so.6(abort+0x121) [0x7fb2ab170535]
 mongod(_ZN5mongo25fassertFailedWithLocationEiPKcj+0x0) [0x560527476739]
 mongod(_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x1DD) [0x560527f498dd]
 mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb+0x8E8) [0x560527f33758]
 mongod(+0x10F49CA) [0x560527f2b9ca]
 mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x69A) [0x560527e1150a]
 mongod(main+0xE9B) [0x560527482bab]
 libc.so.6(__libc_start_main+0xEB) [0x7fb2ab17209b]
 mongod(+0x6AFCFF) [0x5605274e6cff]
-----  END BACKTRACE  -----
2020-02-14T11:42:32.233+0100 I CONTROL  [initandlisten] MongoDB starting : pid=8218 port=39001 dbpath=/srv/public/cryosparc2/cryosparc2_database 64-bit host=gpu031
2020-02-14T11:42:32.233+0100 I CONTROL  [initandlisten] db version v3.4.10-4-g67ee356c6b
2020-02-14T11:42:32.233+0100 I CONTROL  [initandlisten] git version: 67ee356c6be377cda547d16423daef5beb4e8377
2020-02-14T11:42:32.233+0100 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2p  14 Aug 2018
2020-02-14T11:42:32.233+0100 I CONTROL  [initandlisten] allocator: tcmalloc
2020-02-14T11:42:32.233+0100 I CONTROL  [initandlisten] modules: none
2020-02-14T11:42:32.233+0100 I CONTROL  [initandlisten] build environment:
2020-02-14T11:42:32.233+0100 I CONTROL  [initandlisten]     distarch: x86_64
2020-02-14T11:42:32.233+0100 I CONTROL  [initandlisten]     target_arch: x86_64
2020-02-14T11:42:32.233+0100 I CONTROL  [initandlisten] options: { net: { port: 39001 }, replication: { oplogSizeMB: 64, replSet: "meteor" }, storage: { dbPath: "/srv/public/cryosparc2/cryosparc2_database", journal: { enabled: false }, wiredTiger: { engineConfig: { cacheSizeGB: 4.0 } } } }
2020-02-14T11:42:32.261+0100 I -        [initandlisten] Detected data files in /srv/public/cryosparc2/cryosparc2_database created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-02-14T11:42:32.261+0100 I STORAGE  [initandlisten]
2020-02-14T11:42:32.261+0100 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2020-02-14T11:42:32.261+0100 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-02-14T11:42:32.261+0100 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=4096M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),,log=(enabled=false),
2020-02-14T11:42:32.282+0100 E STORAGE  [initandlisten] WiredTiger error (-31802) [1581676952:282924][8218:0x7f71f8981d40], file:sizeStorer.wt, WT_SESSION.open_cursor: unable to read root page from file:sizeStorer.wt: WT_ERROR: non-specific WiredTiger error
2020-02-14T11:42:32.282+0100 I -        [initandlisten] Invariant failure: ret resulted in status UnknownError: -31802: WT_ERROR: non-specific WiredTiger error at src/mongo/db/storage/wiredtiger/wiredtiger_size_storer.cpp 69
2020-02-14T11:42:32.282+0100 I -        [initandlisten]

***aborting after invariant() failure


2020-02-14T11:42:32.286+0100 F -        [initandlisten] Got signal: 6 (Aborted).

 0x5620bcecdd6b 0x5620bceccfe6 0x5620bcecd4c3 0x7f71f8b5e730 0x7f71f89c27bb 0x7f71f89ad535 0x5620bc0ec739 0x5620bcbbf8dd 0x5620bcba9758 0x5620bcba19ca 0x5620bca8750a 0x5620bc0f8bab 0x7f71f89af09b 0x5620bc15ccff
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"5620BBAAD000","o":"1420D6B","s":"_ZN5mongo15printStackTraceERSo"},{"b":"5620BBAAD000","o":"141FFE6"},{"b":"5620BBAAD000","o":"14204C3"},{"b":"7F71F8B4C000","o":"12730"},{"b":"7F71F898B000","o":"377BB","s":"gsignal"},{"b":"7F71F898B000","o":"22535","s":"abort"},{"b":"5620BBAAD000","o":"63F739","s":"_ZN5mongo25fassertFailedWithLocationEiPKcj"},{"b":"5620BBAAD000","o":"11128DD","s":"_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"},{"b":"5620BBAAD000","o":"10FC758","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb"},{"b":"5620BBAAD000","o":"10F49CA"},{"b":"5620BBAAD000","o":"FDA50A","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"5620BBAAD000","o":"64BBAB","s":"main"},{"b":"7F71F898B000","o":"2409B","s":"__libc_start_main"},{"b":"5620BBAAD000","o":"6AFCFF"}],"processInfo":{ "mongodbVersion" : "3.4.10-4-g67ee356c6b", "gitVersion" : "67ee356c6be377cda547d16423daef5beb4e8377", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.19.0-6-amd64", "version" : "#1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11)", "machine" : "x86_64" }, "somap" : [ { "b" : "5620BBAAD000", "elfType" : 3, "buildId" : "A14732EAAA95508462633A5030B501E08B1B106F" }, { "b" : "7FFCCCDBD000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "50CD71D25893F0071A09BC55A2197F59A787ADCB" }, { "b" : "7F71F9571000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0", "elfType" : 3 }, { "b" : "7F71F92FB000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libssl.so.1.0.0", "elfType" : 3 }, { "b" : "7F71F8EBA000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libcrypto.so.1.0.0", "elfType" : 3 }, { "b" : "7F71F8E7F000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "5DCF98AD684962BE494AF28A1051793FD39E4EBC" }, { "b" : "7F71F8E7A000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D3583C742DD47AAA860C5AE0C0C5BDBCD2D54F61" }, { "b" : "7F71F8D04000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libstdc++.so.6", "elfType" : 3 }, { "b" : "7F71F8B81000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "885DDA4B4A5CEA600E7B5B98C1AD86996C8D2299" }, { "b" : "7F71F8B6D000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7F71F8B4C000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "E91114987A0147BD050ADDBD591EB8994B29F4B3" }, { "b" : "7F71F898B000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "18B9A9A8C523E5CFE5B5D946D605D09242F09798" }, { "b" : "7F71F8986000", "path" : "/lib/x86_64-linux-gnu/libutil.so.1", "elfType" : 3, "buildId" : "F44B754DCF4119E5743708FED89461489880474D" }, { "b" : "7F71F9753000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "F25DFD7B95BE4BA386FD71080ACCAE8C0732B711" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x3B) [0x5620bcecdd6b]
 mongod(+0x141FFE6) [0x5620bceccfe6]
 mongod(+0x14204C3) [0x5620bcecd4c3]
 libpthread.so.0(+0x12730) [0x7f71f8b5e730]
 libc.so.6(gsignal+0x10B) [0x7f71f89c27bb]
 libc.so.6(abort+0x121) [0x7f71f89ad535]
 mongod(_ZN5mongo25fassertFailedWithLocationEiPKcj+0x0) [0x5620bc0ec739]
 mongod(_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x1DD) [0x5620bcbbf8dd]
 mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb+0x8E8) [0x5620bcba9758]
 mongod(+0x10F49CA) [0x5620bcba19ca]
 mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x69A) [0x5620bca8750a]
 mongod(main+0xE9B) [0x5620bc0f8bab]
 libc.so.6(__libc_start_main+0xEB) [0x7f71f89af09b]
 mongod(+0x6AFCFF) [0x5620bc15ccff]
-----  END BACKTRACE  -----
2020-02-14T11:42:35.337+0100 I CONTROL  [initandlisten] MongoDB starting : pid=8227 port=39001 dbpath=/srv/public/cryosparc2/cryosparc2_database 64-bit host=gpu031
2020-02-14T11:42:35.338+0100 I CONTROL  [initandlisten] db version v3.4.10-4-g67ee356c6b
2020-02-14T11:42:35.338+0100 I CONTROL  [initandlisten] git version: 67ee356c6be377cda547d16423daef5beb4e8377
2020-02-14T11:42:35.338+0100 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2p  14 Aug 2018
2020-02-14T11:42:35.338+0100 I CONTROL  [initandlisten] allocator: tcmalloc
2020-02-14T11:42:35.338+0100 I CONTROL  [initandlisten] modules: none
2020-02-14T11:42:35.338+0100 I CONTROL  [initandlisten] build environment:
2020-02-14T11:42:35.338+0100 I CONTROL  [initandlisten]     distarch: x86_64
2020-02-14T11:42:35.338+0100 I CONTROL  [initandlisten]     target_arch: x86_64
2020-02-14T11:42:35.338+0100 I CONTROL  [initandlisten] options: { net: { port: 39001 }, replication: { oplogSizeMB: 64, replSet: "meteor" }, storage: { dbPath: "/srv/public/cryosparc2/cryosparc2_database", journal: { enabled: false }, wiredTiger: { engineConfig: { cacheSizeGB: 4.0 } } } }
2020-02-14T11:42:35.365+0100 I -        [initandlisten] Detected data files in /srv/public/cryosparc2/cryosparc2_database created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-02-14T11:42:35.365+0100 I STORAGE  [initandlisten]
2020-02-14T11:42:35.365+0100 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2020-02-14T11:42:35.365+0100 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-02-14T11:42:35.365+0100 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=4096M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),,log=(enabled=false),
2020-02-14T11:42:35.387+0100 E STORAGE  [initandlisten] WiredTiger error (-31802) [1581676955:387402][8227:0x7f7bfbc7ad40], file:sizeStorer.wt, WT_SESSION.open_cursor: unable to read root page from file:sizeStorer.wt: WT_ERROR: non-specific WiredTiger error
2020-02-14T11:42:35.387+0100 I -        [initandlisten] Invariant failure: ret resulted in status UnknownError: -31802: WT_ERROR: non-specific WiredTiger error at src/mongo/db/storage/wiredtiger/wiredtiger_size_storer.cpp 69
2020-02-14T11:42:35.387+0100 I -        [initandlisten]

***aborting after invariant() failure


2020-02-14T11:42:35.391+0100 F -        [initandlisten] Got signal: 6 (Aborted).

 0x55f5f7603d6b 0x55f5f7602fe6 0x55f5f76034c3 0x7f7bfbe57730 0x7f7bfbcbb7bb 0x7f7bfbca6535 0x55f5f6822739 0x55f5f72f58dd 0x55f5f72df758 0x55f5f72d79ca 0x55f5f71bd50a 0x55f5f682ebab 0x7f7bfbca809b 0x55f5f6892cff
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"55F5F61E3000","o":"1420D6B","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55F5F61E3000","o":"141FFE6"},{"b":"55F5F61E3000","o":"14204C3"},{"b":"7F7BFBE45000","o":"12730"},{"b":"7F7BFBC84000","o":"377BB","s":"gsignal"},{"b":"7F7BFBC84000","o":"22535","s":"abort"},{"b":"55F5F61E3000","o":"63F739","s":"_ZN5mongo25fassertFailedWithLocationEiPKcj"},{"b":"55F5F61E3000","o":"11128DD","s":"_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"},{"b":"55F5F61E3000","o":"10FC758","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb"},{"b":"55F5F61E3000","o":"10F49CA"},{"b":"55F5F61E3000","o":"FDA50A","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"55F5F61E3000","o":"64BBAB","s":"main"},{"b":"7F7BFBC84000","o":"2409B","s":"__libc_start_main"},{"b":"55F5F61E3000","o":"6AFCFF"}],"processInfo":{ "mongodbVersion" : "3.4.10-4-g67ee356c6b", "gitVersion" : "67ee356c6be377cda547d16423daef5beb4e8377", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.19.0-6-amd64", "version" : "#1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11)", "machine" : "x86_64" }, "somap" : [ { "b" : "55F5F61E3000", "elfType" : 3, "buildId" : "A14732EAAA95508462633A5030B501E08B1B106F" }, { "b" : "7FFD3C18C000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "50CD71D25893F0071A09BC55A2197F59A787ADCB" }, { "b" : "7F7BFC86A000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/lib/libpython2.7.so.1.0", "elfType" : 3 }, { "b" : "7F7BFC5F4000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libssl.so.1.0.0", "elfType" : 3 }, { "b" : "7F7BFC1B3000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libcrypto.so.1.0.0", "elfType" : 3 }, { "b" : "7F7BFC178000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "5DCF98AD684962BE494AF28A1051793FD39E4EBC" }, { "b" : "7F7BFC173000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D3583C742DD47AAA860C5AE0C0C5BDBCD2D54F61" }, { "b" : "7F7BFBFFD000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libstdc++.so.6", "elfType" : 3 }, { "b" : "7F7BFBE7A000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "885DDA4B4A5CEA600E7B5B98C1AD86996C8D2299" }, { "b" : "7F7BFBE66000", "path" : "/srv/public/cryosparc2/cryosparc2_master/deps/anaconda/bin/../lib/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7F7BFBE45000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "E91114987A0147BD050ADDBD591EB8994B29F4B3" }, { "b" : "7F7BFBC84000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "18B9A9A8C523E5CFE5B5D946D605D09242F09798" }, { "b" : "7F7BFBC7F000", "path" : "/lib/x86_64-linux-gnu/libutil.so.1", "elfType" : 3, "buildId" : "F44B754DCF4119E5743708FED89461489880474D" }, { "b" : "7F7BFCA4C000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "F25DFD7B95BE4BA386FD71080ACCAE8C0732B711" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x3B) [0x55f5f7603d6b]
 mongod(+0x141FFE6) [0x55f5f7602fe6]
 mongod(+0x14204C3) [0x55f5f76034c3]
 libpthread.so.0(+0x12730) [0x7f7bfbe57730]
 libc.so.6(gsignal+0x10B) [0x7f7bfbcbb7bb]
 libc.so.6(abort+0x121) [0x7f7bfbca6535]
 mongod(_ZN5mongo25fassertFailedWithLocationEiPKcj+0x0) [0x55f5f6822739]
 mongod(_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x1DD) [0x55f5f72f58dd]
 mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb+0x8E8) [0x55f5f72df758]
 mongod(+0x10F49CA) [0x55f5f72d79ca]
 mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x69A) [0x55f5f71bd50a]
 mongod(main+0xE9B) [0x55f5f682ebab]
 libc.so.6(__libc_start_main+0xEB) [0x7f7bfbca809b]
 mongod(+0x6AFCFF) [0x55f5f6892cff]
-----  END BACKTRACE  -----