So the past few posts detailed how to parse Portfolio’s logs to find troublesome files as well as how to restart Portfolio’s services.
Well when Portfolio is struggling to catalog files, it can generate massive amounts of temp files & folders, which if not maintained can fill the Portfolio hosts hard drive.
Not a great situation, so I’ve written the below to help automate the maintenance.
As mentioned in my previous post, we’re currently cataloging around 40TB of data in Portfolio.
However, again as per the previous post, we’ve found some files to be troublesome causing issues cataloging. Once Portfolio attempts to catalog these files, the processes used can hang & get stuck on those files.
Portfolio then spawns more & more processes for the next files it finds until the box running Portfolio eventually runs out of resources.
To maintain the hosts uptime, we’re using the below script to restart the services daily.
As mentioned before, we use Portfolio to archive old projects.
However since moving to Portfolio v1.x (from v11), we’ve had numerous issues. The main one has been getting the 40TB of data we host re-cataloged into Portfolio.
I’ve recently deep dived into Portfolio, & found some files that cause issues & these can be found via the logs. Parsing them allows me to “fix” the troublesome files or delete any corrupt ones.
Below is the how I’m parsing the logs.
For a number of years we have used Portfolio as an archive server for old projects to be archived from our various data servers.
This would free up space on those servers, & with the added benefits of using the built-in meta-tagging ability to view cataloged files via a web browser.
With the release of the most recent version, we moved from a Windows 2008r2 VM which was starting to creak under the 20+TB of data we had archived to a Mac Pro 64GB RAM, 1TB SSD & 2x24TB LaCie 8big Rack Thunderbolt 2.
However, a few months in & not all data had been cataloged & the Mac Pro was pretty much idling. So I had a hunt through Portfolio’s various files with Extensis support on the phone & came up with the below locations in which the memory allocation can be amended from the defaults to improve performance by utilising more resources.