We currently face a lot of flakiness with our Saucelabs job. These randomly exceed the 2min establish timeout because something throws
off `sauce-connect` in a long-lasting loop where it tries to connect to some of their Saucelabs servers. The initial assumption from the Saucelabs
team was that we might have some invalid firewall rules, but this does not answer why this happens _randomly_, so the latest update from the support
is that there have been some changes in the latest version of `sauce-connect` version that cause this flakiness.
* build: increase saucelabs connect timeout
At some point we went down from a 2min wait timeout to 30seconds because we re-ran the startup script if something failed. Since re-running didn't really help and we removed it, we should move the timeout back to a longer period.
e.g. see: https://circleci.com/gh/angular/material2/25254 where it successfully acquired a tunnel, but just didn't have enough time to actually establish it.
* Switch to rewritten saucelabs version
* Update to latest saucelabs launcher version
* Currently it can happen that our tunnel identifier conflicts with other tunnel identifiers from other Angular projects. A few PRs reported that their tunnel couldn't be re-opened because it was already used and closed successfully.
* No longer runs e2e tests inside of Saucelabs because Chrome is also installed on Travis CI and we should keep Saucelabs available for other browsers that don't run inside of Dockers or Travis.
* Updates the Saucelabs Connect Tunnel to the most recent version that seems to include a major bug fix that should improve tunnel stability
* Changes the CircleCI resource class to the `xlarge` (similar as in angular/angular). This should speed up the Bazel builds due to more CPU threads and more RAM.
* No longer just runs the tests against the first browser of a test platform (travis, saucelabs, browserstack)
* Stability is more valuable than speed (runs sequentially on CI now)
* Update Saucelabs Connect
* Remove unused spec reporter
* Update from Edge 14 to Edge 17
* Fix pixel deviations in Edge 17
* Changes the wait timeout for Saucelabs and Browserstack from 2 minutes to 1 minute. This is possible now because if the timeout fails we still re-try to create a connection a second time.
* The logfile output should be printed if the tunnel connection can't be established. The logging is also useful to see the update message from Saucelabs for example.
* Updates the version of Sauce Connect to avoid potential flakes within the Saucelabs tunnels
* Fixes the invalid path for the `deploy-dashboard` script that runs in the deploy build stage
* Suppresses normal log messages from Saucelabs Conect. Error messages will still show up (In a follow-up PR the normal stdout will be printed together with the stdout on CI failures)
* No longer suppresses the whole Browserstack Tunnel execution. Stderr output will be printed if present. (In a follow-up stdout will be printed as well on CI failures)
* Switches to best-practice variable referencing in Bash
* Updates the saucelabs tunnel binaries to fix CI flakiness for unit tests and protractor.
From what we experience, older versions of Saucelabs tend to cause flakiness immediately (See `master` branch)
Paul Gschwendtner