> Each Lambda invocation executed a simple TruffleHog scan command with concurrency set to 1000. This setup allowed me to complete the scan of 5,600,000 repositories in just over 24 hours.
Gitlab must have been thrilled about a bot cloning 5.6 million repo's in 24 hours. That doesn't really sound responsible to me.
That's 64 clones per second. That's quite a lot but it seems like something that a forge operating at the scale of GitHub can handle, especially if they were --depth=1 (which might have missed some secrets if someone was lazy about clearing their git history).
Provided someone told GitLab Support. This was likely fine. GitLab can handle this much load. The platform as a whole has increased and improved over the years as new customers are added.
Think about this… every CI/CD Job runs a clone. That’s a lot..
"Google Cloud Platform (GCP) credentials were the most leaked secret type on GitLab repositories"
Not surprising, Google SDK are sucking so much in term of authentication.
It's never something simple like an API key, always a shitty iam like opaque function based on an opaque sdk needing to be installed that in the end requires a huge json.
And most of the time, it is a pain in the ass to provide the token "as-is" in a buffer but the sdk expects that you give a file path to it.
So, I easily guess that a lot of lazy devs will just store the credential json file in their project and consider it a job done.
The post keeps saying "verified secrets" - how are they verified? Did the author attempt to login to each service? Or does verified just means that it looks like a valid token?
> Each Lambda invocation executed a simple TruffleHog scan command with concurrency set to 1000. This setup allowed me to complete the scan of 5,600,000 repositories in just over 24 hours.
Gitlab must have been thrilled about a bot cloning 5.6 million repo's in 24 hours. That doesn't really sound responsible to me.
That's 64 clones per second. That's quite a lot but it seems like something that a forge operating at the scale of GitHub can handle, especially if they were --depth=1 (which might have missed some secrets if someone was lazy about clearing their git history).
Provided someone told GitLab Support. This was likely fine. GitLab can handle this much load. The platform as a whole has increased and improved over the years as new customers are added.
Think about this… every CI/CD Job runs a clone. That’s a lot..
Gitlab*
If they don’t like, they will apply rate limiting? Assuming they were well behaved (user agent, IPs).
I also thought the sleep(0.03) was cute. Some well deserved rest for the server to avoid hammering it.
"Google Cloud Platform (GCP) credentials were the most leaked secret type on GitLab repositories"
Not surprising, Google SDK are sucking so much in term of authentication. It's never something simple like an API key, always a shitty iam like opaque function based on an opaque sdk needing to be installed that in the end requires a huge json. And most of the time, it is a pain in the ass to provide the token "as-is" in a buffer but the sdk expects that you give a file path to it. So, I easily guess that a lot of lazy devs will just store the credential json file in their project and consider it a job done.
The post keeps saying "verified secrets" - how are they verified? Did the author attempt to login to each service? Or does verified just means that it looks like a valid token?
Tools like TruffleHog[1] will attempt to verify any credentials it finds by making some sort of authenticated request.
[1] https://github.com/trufflesecurity/trufflehog#validation-
9000 in bounties for 17,000 secrets?
You could make as much in a month creating those vulnerabilities
Truffle Security treasury dollars: There are dozens of us! Dozens!