Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SMB folder does not get synced correctly (failed sync child node) #368

Closed
raffis opened this issue Aug 20, 2019 · 3 comments
Closed

SMB folder does not get synced correctly (failed sync child node) #368

raffis opened this issue Aug 20, 2019 · 3 comments
Labels
bug hasfix storage

Comments

@raffis
Copy link
Contributor

raffis commented Aug 20, 2019

Describe the bug

2019-20-08 10:08:23 [Balloon\Hook,DEBUG]: found registered hook, execute [Balloon\Hook\AutoCreateUser::preDeleteFile]  
2019-20-08 10:08:23 [Balloon\Async\WorkerFactory,WARNING]: Couldn't delete smb://fs001.mbazh.ch/KZU-Intranet-Dokumente/Sekretariat/ABSCHL~1/01AUSZ~1/Wj_2019_20_Noten_N%20S.pdf: Permission denied in /usr/share/balloon/vendor/icewind/smb/src/Native/NativeState.php:148  
2019-20-08 10:08:23 [Balloon\Filesystem\Node\File,ERROR]: failed delete file node [5d506748d715fd000764b7a5]  [object] (Icewind\SMB\Exception\ForbiddenException(code: 13): Invalid request for /Sekretariat/ABSCHL~1/01AUSZ~1/Wj_2019_20_Noten_N%20S.pdf (ForbiddenException) at /usr/share/balloon/vendor/icewind/smb/src/Exception/Exception.php:30)
[stacktrace]
#0 /usr/share/balloon/vendor/icewind/smb/src/Native/NativeState.php(62): Icewind\SMB\Exception\Exception::fromMap(Array, 13, '/Sekretariat/AB...')
#1 /usr/share/balloon/vendor/icewind/smb/src/Native/NativeState.php(74): Icewind\SMB\Native\NativeState->handleError('/Sekretariat/AB...')
#2 /usr/share/balloon/vendor/icewind/smb/src/Native/NativeState.php(150): Icewind\SMB\Native\NativeState->testResult(false, 'smb://fs001.mba...')
#3 /usr/share/balloon/vendor/icewind/smb/src/Native/NativeShare.php(172): Icewind\SMB\Native\NativeState->unlink('smb://fs001.mba...')
#4 /usr/share/balloon/src/lib/Filesystem/Storage/Adapter/Smb.php(159): Icewind\SMB\Native\NativeShare->del('/Sekretariat/AB...')
#5 /usr/share/balloon/src/lib/Filesystem/Node/File.php(532): Balloon\Filesystem\Storage\Adapter\Smb->forceDeleteFile(Object(Balloon\Filesystem\Node\File))
#6 /usr/share/balloon/src/lib/Filesystem/Node/File.php(269): Balloon\Filesystem\Node\File->_forceDelete()
#7 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(913): Balloon\Filesystem\Node\File->delete(true, '5d5bc69633af4', false)
#8 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(742): Balloon\Filesystem\Node\Collection->Balloon\Filesystem\Node\{closure}(Object(Balloon\Filesystem\Node\File))
#9 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(914): Balloon\Filesystem\Node\Collection->doRecursiveAction(Object(Closure), 2)
#10 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(354): Balloon\Filesystem\Node\Collection->_forceDelete('5d5bc69633af4', false)
#11 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(913): Balloon\Filesystem\Node\Collection->delete(true, '5d5bc69633af4', false)
#12 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(742): Balloon\Filesystem\Node\Collection->Balloon\Filesystem\Node\{closure}(Object(Balloon\Filesystem\Node\Collection))
#13 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(914): Balloon\Filesystem\Node\Collection->doRecursiveAction(Object(Closure), 2)
#14 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(354): Balloon\Filesystem\Node\Collection->_forceDelete('5d5bc69633af4', true)
#15 /usr/share/balloon/src/lib/Async/SmbScanner.php(154): Balloon\Filesystem\Node\Collection->delete(true)
#16 /usr/share/balloon/src/lib/Async/SmbScanner.php(226): Balloon\Async\SmbScanner->deleteNode(Object(Balloon\Filesystem\Node\Collection), Object(Balloon\Filesystem\Storage\Adapter\Blackhole))
#17 /usr/share/balloon/src/lib/Async/SmbScanner.php(213): Balloon\Async\SmbScanner->recursiveIterator(Object(Balloon\Filesystem\Node\Collection), Object(Balloon\Filesystem\Node\Collection), Object(Icewind\SMB\Native\NativeShare), Object(Balloon\Filesystem\Storage\Adapter\Blackhole), Object(Balloon\Server\User), Object(Balloon\Filesystem\Storage\Adapter\Smb), '/Sekretariat', true, 1)
#18 /usr/share/balloon/src/lib/Async/SmbScanner.php(100): Balloon\Async\SmbScanner->recursiveIterator(Object(Balloon\Filesystem\Node\Collection), Object(Balloon\Filesystem\Node\Collection), Object(Icewind\SMB\Native\NativeShare), Object(Balloon\Filesystem\Storage\Adapter\Blackhole), Object(Balloon\Server\User), Object(Balloon\Filesystem\Storage\Adapter\Smb), '/', true, 1)
#19 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Worker.php(574): Balloon\Async\SmbScanner->start()
#20 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Worker.php(482): TaskScheduler\Worker->executeJob(Array)
#21 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Worker.php(333): TaskScheduler\Worker->processJob(Array)
#22 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Worker.php(227): TaskScheduler\Worker->queueJob(Array)
#23 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/WorkerManager.php(287): TaskScheduler\Worker->processAll()
#24 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/WorkerManager.php(473): TaskScheduler\WorkerManager->spawnWorker()
#25 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/WorkerManager.php(332): TaskScheduler\WorkerManager->handleJob(Array)
#26 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/WorkerManager.php(176): TaskScheduler\WorkerManager->main()
#27 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Queue.php(148): TaskScheduler\WorkerManager->process()
#28 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Queue.php(93): TaskScheduler\Queue->initWorkerManager()
#29 /usr/share/balloon/src/app/Balloon.App.Cli/Console/Jobs.php(83): TaskScheduler\Queue->process()
#30 /usr/share/balloon/src/lib/Bootstrap/Cli.php(91): Balloon\App\Cli\Console\Jobs->__invoke()
#31 /usr/share/balloon/src/lib/Bootstrap/Cli.php(115): Balloon\Bootstrap\Cli->executeCommand(Object(Balloon\App\Cli\Console\Jobs))
#32 /usr/share/balloon/src/lib/Bootstrap/Cli.php(76): Balloon\Bootstrap\Cli->routeCommand()
#33 /usr/share/balloon/bin/console/ballooncli(26): Balloon\Bootstrap\Cli->process()
#34 {main}
2019-20-08 10:08:23 [Balloon\Async\SmbScanner,ERROR]: failed sync child node [/Sekretariat] in smb mount  [object] (Icewind\SMB\Exception\ForbiddenException(code: 13): Invalid request for /Sekretariat/ABSCHL~1/01AUSZ~1/Wj_2019_20_Noten_N%20S.pdf (ForbiddenException) at /usr/share/balloon/vendor/icewind/smb/src/Exception/Exception.php:30)
[stacktrace]
#0 /usr/share/balloon/vendor/icewind/smb/src/Native/NativeState.php(62): Icewind\SMB\Exception\Exception::fromMap(Array, 13, '/Sekretariat/AB...')
#1 /usr/share/balloon/vendor/icewind/smb/src/Native/NativeState.php(74): Icewind\SMB\Native\NativeState->handleError('/Sekretariat/AB...')
#2 /usr/share/balloon/vendor/icewind/smb/src/Native/NativeState.php(150): Icewind\SMB\Native\NativeState->testResult(false, 'smb://fs001.mba...')
#3 /usr/share/balloon/vendor/icewind/smb/src/Native/NativeShare.php(172): Icewind\SMB\Native\NativeState->unlink('smb://fs001.mba...')
#4 /usr/share/balloon/src/lib/Filesystem/Storage/Adapter/Smb.php(159): Icewind\SMB\Native\NativeShare->del('/Sekretariat/AB...')
#5 /usr/share/balloon/src/lib/Filesystem/Node/File.php(532): Balloon\Filesystem\Storage\Adapter\Smb->forceDeleteFile(Object(Balloon\Filesystem\Node\File))
#6 /usr/share/balloon/src/lib/Filesystem/Node/File.php(269): Balloon\Filesystem\Node\File->_forceDelete()
#7 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(913): Balloon\Filesystem\Node\File->delete(true, '5d5bc69633af4', false)
#8 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(742): Balloon\Filesystem\Node\Collection->Balloon\Filesystem\Node\{closure}(Object(Balloon\Filesystem\Node\File))
#9 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(914): Balloon\Filesystem\Node\Collection->doRecursiveAction(Object(Closure), 2)
#10 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(354): Balloon\Filesystem\Node\Collection->_forceDelete('5d5bc69633af4', false)
#11 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(913): Balloon\Filesystem\Node\Collection->delete(true, '5d5bc69633af4', false)
#12 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(742): Balloon\Filesystem\Node\Collection->Balloon\Filesystem\Node\{closure}(Object(Balloon\Filesystem\Node\Collection))
#13 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(914): Balloon\Filesystem\Node\Collection->doRecursiveAction(Object(Closure), 2)
#14 /usr/share/balloon/src/lib/Filesystem/Node/Collection.php(354): Balloon\Filesystem\Node\Collection->_forceDelete('5d5bc69633af4', true)
#15 /usr/share/balloon/src/lib/Async/SmbScanner.php(154): Balloon\Filesystem\Node\Collection->delete(true)
#16 /usr/share/balloon/src/lib/Async/SmbScanner.php(226): Balloon\Async\SmbScanner->deleteNode(Object(Balloon\Filesystem\Node\Collection), Object(Balloon\Filesystem\Storage\Adapter\Blackhole))
#17 /usr/share/balloon/src/lib/Async/SmbScanner.php(213): Balloon\Async\SmbScanner->recursiveIterator(Object(Balloon\Filesystem\Node\Collection), Object(Balloon\Filesystem\Node\Collection), Object(Icewind\SMB\Native\NativeShare), Object(Balloon\Filesystem\Storage\Adapter\Blackhole), Object(Balloon\Server\User), Object(Balloon\Filesystem\Storage\Adapter\Smb), '/Sekretariat', true, 1)
#18 /usr/share/balloon/src/lib/Async/SmbScanner.php(100): Balloon\Async\SmbScanner->recursiveIterator(Object(Balloon\Filesystem\Node\Collection), Object(Balloon\Filesystem\Node\Collection), Object(Icewind\SMB\Native\NativeShare), Object(Balloon\Filesystem\Storage\Adapter\Blackhole), Object(Balloon\Server\User), Object(Balloon\Filesystem\Storage\Adapter\Smb), '/', true, 1)
#19 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Worker.php(574): Balloon\Async\SmbScanner->start()
#20 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Worker.php(482): TaskScheduler\Worker->executeJob(Array)
#21 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Worker.php(333): TaskScheduler\Worker->processJob(Array)
#22 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Worker.php(227): TaskScheduler\Worker->queueJob(Array)
#23 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/WorkerManager.php(287): TaskScheduler\Worker->processAll()
#24 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/WorkerManager.php(473): TaskScheduler\WorkerManager->spawnWorker()
#25 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/WorkerManager.php(332): TaskScheduler\WorkerManager->handleJob(Array)
#26 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/WorkerManager.php(176): TaskScheduler\WorkerManager->main()
#27 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Queue.php(148): TaskScheduler\WorkerManager->process()
#28 /usr/share/balloon/vendor/gyselroth/mongodb-php-task-scheduler/src/Queue.php(93): TaskScheduler\Queue->initWorkerManager()
#29 /usr/share/balloon/src/app/Balloon.App.Cli/Console/Jobs.php(83): TaskScheduler\Queue->process()
#30 /usr/share/balloon/src/lib/Bootstrap/Cli.php(91): Balloon\App\Cli\Console\Jobs->__invoke()
#31 /usr/share/balloon/src/lib/Bootstrap/Cli.php(115): Balloon\Bootstrap\Cli->executeCommand(Object(Balloon\App\Cli\Console\Jobs))
#32 /usr/share/balloon/src/lib/Bootstrap/Cli.php(76): Balloon\Bootstrap\Cli->routeCommand()
#33 /usr/share/balloon/bin/console/ballooncli(26): Balloon\Bootstrap\Cli->process()
#34 {main}

To Reproduce

  1. Create smb folder a
  2. Create file /a/test.txt
  3. Stop smb job listener
  4. delete folder from the smb share
  5. start jobs, manually trigger smb sync (currently the only possible way, see Trigger resync of smb share (folder) #367 ):
    db.taskscheduler.jobs.insert({"class" : "Balloon\\Async\\SmbScanner","status" : NumberInt(0),"created" : ISODate("2019-08-20T10:51:22.670Z"),"started" : ISODate("2019-08-19T19:51:23.682Z"),"ended" : ISODate("2019-08-19T19:51:22.670Z"),"worker" : ObjectId("5d5410df2dab1800065a08ce"),"data" : {"id" : ObjectId("5d5bdbb2a85d38000d7af494")},"options" : {"at" : NumberInt(0),"interval" : NumberInt(0),"retry" : NumberInt(0),"retry_interval" : NumberInt(300),"force_spawn" : false,"timeout" : NumberInt(0),"ignore_data" : false}})
  6. folder gets removed via Adapter\Smb instead Adapter\Blackhole.

Expected behavior

All actions especially removals executed by SmbScanner must be executed using the Blackhole storage adapter.
This does not work currently as the blackhole adapter is set but each new node load via the vfs does load each parent separately meaning a new parent instance for each load which does not have the previously set blackhole storage adapter set on an another instance of the same node id.

Environment

  • balloon server version: v2.6.2
  • Deployed as: [e.g. docker,deb package,tar,compiled manually]
  • PHP version: [e.g. 7.2.1]
  • MongoDB version: [e.g. v3.4.18]

Additional context

Add any other context about the problem here.
@raffis raffis closed this as completed in 7aba98a Aug 21, 2019
@raffis raffis reopened this Aug 22, 2019
@raffis
Copy link
Contributor Author

raffis commented Aug 22, 2019

This still leads to folders out of sync:
2019-22-08 07:06:12 [Balloon\Async\SmbScanner,ERROR]: failed sync child node [/Sekretariat] in smb mount [object] (Icewind\SMB\Exception\ForbiddenException(code: 13): Invalid request for /Sekretariat/ABSCHL~1/01AUSZ~1/Wj_2019_20_Noten_N%20S.pdf (ForbiddenException) at /usr/share/balloon/vendor/icewind/smb/src/Exception/Exception.php:30)

@raffis raffis removed the hasfix label Aug 22, 2019
@raffis
Copy link
Contributor Author

raffis commented Aug 22, 2019

It is not exactly clear how folders like "ABSCHL~1" made it into an smblist alongside the original name, those are windows shortcut names visible by executing dir /x on windows.

That said balloon still uses the Adapter\Smb instead Adapter\Blackhole to execute removal operations.

@raffis
Copy link
Contributor Author

raffis commented Aug 22, 2019 •
edited
Loading

511d8ed now enables balloon to always use the storage from parent collections if a different one other than the default is set.

@raffis raffis added the hasfix label Aug 22, 2019
@raffis raffis closed this as completed Aug 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug hasfix storage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@raffis