Coavmi

coavmi

The Surprisingly Weird Tech Making Drones Fly Smarter and Students Learn Faster

On Combinations Amongst Masters or Workmen against Each Other

CALCULATING MACHINES

You launched your service and rapidly onboarding customers. You're moving fast, repeatedly deploying one new feature after another. But with the uptick in releases, bugs are creeping in, and you're finding yourself having to troubleshoot, rollback, squash bugs, and then redeploy changes. Moving fast but breaking things. What can you do to quickly detect issues — before your customers report them? By Using Canaries. In this post, you'll learn about the concept of canaries, example code, best practices, and other considerations, including both maintenance and financial implications with running them. What is a canary Source: grass-lifeisgood/Shutterstock Back in the early 1900s, canaries were used by miners for . detecting carbon monoxide and other dangerous gases Miners would bring their canaries down with them to the coalmine, and when their canary stopped chirping, it was time for everyone to evacuate immediately. In the context of computing systems, canaries perform end-to-end testing, aiming to exercise the entire software stack of your application: they behave like your end-users, emulating customer behavior. The canaries are just pieces of software that are always running and constantly monitoring the state of your system; they emit metrics into your monitoring system (more discussion on monitoring in a separate post), which then triggers an alarm when some defined threshold breaches. What do canaries offer? They answer the question: "Is my service running?" More sophisticated canaries can offer a deeper look into your service. Instead of canaries just emitting a binary 1 or 0 — up or down — they can be designed such that they emit more meaningful metrics that measure latency from the client's perspective. First steps with building your canary If you don't have any canaries running that monitor your system, you don't necessarily have to start with rolling your own. Your first canary can require little to no code. One way to gain immediate visibility into your system would be to use synthetic monitoring services such as BetterUptime or PingDom, or StatusCake. These services offer a web interface, allowing you to configure HTTP(s) endpoints that their canaries will periodically poll. When their systems detect an issue (e.g., TCP connection failing, bad HTTP response), they can send you email or text notifications. Or, if your systems are deployed in Amazon Web Services, you can write Python or Node scripts that . integrate with CloudWatch But if you are interested in developing your own custom canaries that do more than a simple probe, read on. Where to begin Remember, canaries should behave just like real customers. Your customer might be a real human being or another piece of software. Regardless of the type of customer, you'll want to start simple. Similar to the managed services described above, your first canary should start with emitting a simple metric into your monitoring system, indicating whether the endpoint is up or down. For example, if you have a web service, perform a vanilla HTTP GET. When successful, the canary will emit and under failure, . http_get_homepage_success=1 http_get_homepage_success=0 Example canary - monitoring cache layer Imagine you have a simple key/value store system that serves as a caching layer. To monitor this layer, every minute, our canary will: 1) perform a write 2) perform a read 3) validate the response. ( ):

    successful_run = :
        put_response = cache_put( , )
        write_successful = put_response == Publish_metric( , write_successful)
        value = cache_get( )
        successful_read = value = publish_metric( , is_successful_read)
        canary_successful_run = Except error:
        log_exception( % error)
    Finally:
        Publish_metric( , int(successful_run))
        sleep_for_in_seconds = sleep(sleep_for_in_seconds) while True False try 'foo' 'bar' 'OK' 'cache_engine_successful_write' 'foo' 'bar' 'cache_engine_successful_read' True as "Canary failed due to error: %s" 'cache_engine_canary_successful_run' 60 Cache engine failure during deployment With this canary in place emitting metrics, we might then choose to integrate the canary with our code deployment pipeline. In the example below, I triggered a code deployment (riddled with bugs), and the canary detected an issue, triggering an automatic rollback: Best Practices The above code example was very unsophisticated, and you'll want to keep the following best practices in mind: . Although a good canary should test different behaviors/states of your system, they should in no way interfere with the real user experience. That is, their side effects should be self-contained. The canaries should NOT interfere with the real user experience . Ideally, the canary runs frequently (e.g., every 15 seconds, every 1 minute). They should always be on, always running, and should be testing at regular intervals . If your alarms fire off on a single data point, you increase the likelihood of false alarms, engaging your service teams unnecessarily. The alarms that you create when your canary reports an issue should only trigger off more than one datapoint . Essentially, the deployment system should monitor the metrics that the canary emits, and if an error is detected for more than N minutes, the deployment should automatically rollback (more of the safety of automated rollbacks in a separate post). Integrate the canary into your continuous integration/continuous deployment pipeline . Success criteria should be more than verifying that the HTTP status code is a 200 OK. If your web services return payload in the form of JSON, analyze the payload and verify that it's both syntactically and semantically correct. When rolling your own canary, do more than just inspect the HTTP headers Cost of canaries Of course, canaries are not free. Regardless of whether or not you rely on a third-party service or roll your own, you'll need to be aware of the maintenance and financial costs. Maintenance A canary is just another piece of software. The underlying implementation might be just a few bash scripts cobbled together or a full-blown client application. In either case, you need to maintain them just like any other code package. Financial costs How often is the canary running? How many instances of the canary are running? Are they geographically distributed to test from different locations? These are some of the questions that you must ask since they impact the cost of running them. Beyond canaries When building systems, you want a canary that behaves like your customer, one that allows you to quickly detect issues as soon as your service(s) chokes. If you are vending an API, then your canary should exercise the different URIs. If you are testing the front end, then your canary can be programmed to mimic a customer using a browser using libraries such as selenium. The canaries are a great place to start if you are just launching a service. But there's a lot more work required to create an operationally robust service. You'll want to inject failures into your system. Also, you need a crystal clear understanding of how your system should behave when its dependencies fail. These are some of the topics that I'll cover in the next series of blog posts. Let's Connect Let's connect and talk more about software and DevOps. Follow me on Twitter: @memattchung Previously published on https://blog.mattchung.me/2021/06/21/is-my-service-up-and-running-canaries-to-the-rescue/ .

Canaries to the Rescue: Catching Service Issues Before the End User

This Slack discussion by , , and occurred in 's official #slogging-beta channel, and has been edited for readability. Arthur Limarc Austin Richard-Kubina Slogging How scared you are of deepfakes? Arthur Nov 6, 2020, 3:23 PM It's tricking my brain Arthur Nov 6, 2020, 3:23 PM I am scared in the sense that I know there are a lot of people who will easily be fooled by them. Many people (Americans) already blindly trust what politicians say without checking the facts. Imagine if disinformation had a very credible deepfake video to go with it? I know we will have deepfake-detector technology in the future made by the likes of Twitch and Youtube to flag videos as fake. But until then, I’m worried people can use them for disinformation campaigns. Limarc Nov 6, 2020, 3:38 PM While YouTube will have anti-deepfake algo soon or later, I think their main goal will be to protect their ad revenue. And it will generate a next level of censorship on this platform. But at least it will reduce some problems. Arthur Nov 6, 2020, 3:46 PM @Limarc From the other side. What if I'll made a deepfake video and will share it without uploading on big platforms? like via AirDrop or Protonmail I mean we still can't figure out how to remove spam calls on our phones... Or how to protect our children from "bad websites" Arthur Nov 6, 2020, 3:46 PM agree with on the above, but also wanted to add: what i also worry about is the eventual cat-and-mouse game with deepfakes, like we have with ads/ad blockers and malware/anti-malware austin Nov 6, 2020, 3:46 PM @Limarc those last examples are great examples of the cat and mouse game we're going to see between deepfakes and the prevention tech austin Nov 6, 2020, 3:47 PM @Arthur eventually cryptographic signing could end it, but...that's assuming we have a good PKI in place, and good software that hides the cryptography. no one gives a shit about a signature, they just want that "verified" checkmark austin Nov 6, 2020, 3:48 PM about cryprographic sign. i think we already must have and use those things. But we still have tons of passwords for different websites. Arthur Nov 6, 2020, 3:52 PM @Austin what i'm describing isn't so much to do with passwords. what i'm describing is a special cryptographic signature on a video, by the creator, that verifies that person made that video and it wasn't tampered with. we have things like sha/md5 hashes, but those are geared more for developers (and say nothing about who authored it, whether the website/source was tampered with, etc) austin Nov 6, 2020, 3:54 PM @Arthur can't really just have a "verified" checkmark though either, right? seems like you'd just be kicking the can down the road.  you'd need to trust your own software to run the verification yourself too. richard-kubina Nov 6, 2020, 3:56 PM well that's what i mean by having a reliable PKI in place. if anyone with the knowhow can verify the signature, and you can get a reliable public key from the internet, then any trickery on the software/website/whatever's showing the checkmark would be discovered very quickly. you could even write software as a sort of "canary in the coalmine", an alert if someone's signature underneath the checkmark doesn't match austin Nov 6, 2020, 3:58 PM I get it. Especially if it will be incorporated into major tools, but controlled by an outside company that will "verify" that everything is protected. Arthur Nov 6, 2020, 3:59 PM @Austin What I mean that while we have power to get rid of login forms - we still using them and exposing everything. Another thing, for protection - it probably will be controlled by some monopoly. Like Alphabet bought ReCaptcha years ago. Arthur Nov 6, 2020, 3:59 PM what i mean to do here is make it less of a cat and mouse game like ad blocking/detection, and more of a cryptography problem that's much harder to crack for the adversary, the person or people doing the tampering right, so i'm picturing more of a distributed PKI infra, but that's probably a pipe dream 😕 a few big companies could do it though. as for logins: biometrics ftw. kill the password, i totally agree austin Nov 6, 2020, 4:00 PM @Richard-Kubina @Arthur yea, the more layers the harder it is to fake.  Just thinking you also need to consider your own machine might not be trust worthy!  Like we saw with the Meltdown and Spectre vulnerability.  :tinfoil-hat:  Might end up seeing a happy green checkmark that shouldn't be there.  I think you will always have this cat and mouse game, it just comes down to how lucrative it is to crack or how high the stakes are to put up the fight.  I am definitely not a cryptography expert though... richard-kubina Nov 6, 2020, 4:10 PM oh damn 😱 you've got me thinking about the "management engine" inside the vast majority of PCs now... :tinfoil-hat: you've got me there, though. ultimately it would take a lot of consensus, on the part of verifiers, and it's still a cat and mouse game...i thought it would be harder to crack, and it is, if and only if the author's machine isn't compromised 🙃 and high profile targets like public figures, would prooobably get hacked. i don't get how people work in computer security without going insane so it's just turtles all the way down, then? can we just throw computers away and start over because i have thoughts on that austin Nov 6, 2020, 4:28 PM @Richard-Kubina haha naw I don't mean to win an argument, I just figure there's always that risk of not being able to trust the handy ✅ .  like with Apple approving malware in their store richard-kubina Nov 6, 2020, 4:33 PM https://techcrunch.com/2020/08/31/apple-notarized-mac-malware/ oh wow @ the malware thing....did not hear about that one 👀 but nah, not arguing here, i'm just genuinely curious! like...is it possible to ensure security, and if so, what's the confidence level? how much consensus is "enough"? austin Nov 6, 2020, 4:34 PM with all the layers of current computers, i'm honestly not sure austin Nov 6, 2020, 4:35 PM yea, there were other ones with microsoft's windows updater that was downloading malware..  I forget which software download site also was compromised and serving up executables with malware.. https looked fine at point.. but it didn't matter richard-kubina Nov 6, 2020, 4:35 PM I do like the idea of 'consensus' though... richard-kubina Nov 6, 2020, 4:38 PM would be nice to have security experts weigh in here! 👋 richard-kubina Nov 6, 2020, 4:38 PM oh god... you're killing me here man 😂 i'm gonna wind up making software for little embedded chips that don't talk to the internet just to be safe... it's a great point though. so how sure can we be that certain systems aren't already compromised?? like...what percentage of consumer PCs are vulnerable to attack if you do something simple like click a link? i'd wager close to 98-99% but not 100% austin Nov 6, 2020, 4:38 PM yeah lots of "what ifs" here haha 😅 would be very nice to get an expert opinion austin Nov 6, 2020, 4:39 PM i guess going back to deep fakes, it's a people problem and not a tech problem, right?  you just gotta double check everything.. if someone emails you or calls you sounding like your boss telling you to send money somewhere, you may wanna verify that on another channel.. sorta like two-factor ...verification.  Get on your landline and call them up ☎ and hope you aren't getting phreaked!!1 😅 richard-kubina Nov 6, 2020, 4:43 PM two factor is a sorta consensus, that not all lines of communication are compromised .. better odds richard-kubina Nov 6, 2020, 4:44 PM Sorry for the off-topic. For two years in our national tech university, I have only two teachers who did something real in terms of our "coding profession". One teacher created cool simulation software for meteorites that falling on earth and breaking down into small pieces. I mean, like real software that was used. The second teacher was the youngest professor in our division. He actually creates some extension to one of the big crypto-algos(like SHA or something). He pissed off old professors because he gets grants money from local coding companies and has a cool separate computer lab. I'm sure it's affected me somehow... Arthur Nov 6, 2020, 4:44 PM richard-kubina Nov 6, 2020, 4:44 PM It's interesting to re-read articles from 2019 about deepfakes technology. like 2 years time-machine https://hackernoon.com/why-do-we-need-a-solution-to-deepfake-b81f3yi7 https://hackernoon.com/the-deepfake-challenge-nyf1k31kn https://hackernoon.com/the-light-side-of-deepfakes-how-the-technology-can-be-used-for-good-4hr32pp https://www.forbes.com/sites/johnkoetsier/2019/07/17/viral-app-faceapp-now-owns-access-to-more-than-150-million-peoples-faces-and-names/amp/ https://www.digitaltrends.com/news/faceapp-photos-privacy-terms-of-service/ Arthur Jan 13, 2021, 10:08 PM Here is a collection of cool deepfakes, that available on YouTube. it's also easy to see how technology evolving: not only editing, capturing, but ML algorithms can do much more work, even with smaller datasets Arthur Jan 13, 2021, 10:14 PM https://www.youtube.com/watch?v=NbjE5wXT3RE Also check our Deepfakes tag page: https://hackernoon.com/tagged/deepfakes

Slogging #26: How Scared You Are Of Deepfakes?

How twisted can things get when an anonymous piece of code harms people but can’t be stopped? Two cinematic masterpieces offer us devastating answers and lots of food for thought: . In this Stanley Kubrick classic that takes place at the height of the cold war, an American General loses his mind and orders a nuclear strike on the Soviets. The planes are mid-air, will drop the bombs in just 2 hours and cannot be stopped by the anxious American government. Once the Americans lose all hope of preventing the strike, the president calls the Soviets to warn them and offers the planes’ positions and targets. Sadly, the Soviet ambassador then reveals that the Soviet Union has long been running a computerized “doomsday device”: upon nuclear strike on Russia, the device is programmed to automatically detonate an array of highly radioactive bombs that will wipe out life on earth. The device can’t be switched off. In fact, it’s designed to go off if anyone tries to tamper with it Dr. Strangelove (1965) any all . is probably Black Mirror’s most disturbing episode. As summarized by Wikipedia: . Shut Up And Dance is a breathtaking tour-de-force of what Black Mirror is all about: dark, uncomfortable yet highly recommended watch Black Mirror S03 E03 (2016) Shut Up And Dance it tells the story of a teenage boy who is blackmailed into committing bizarre and criminal acts by a mysterious hacker possessing a video of him masturbating. The boy is joined by a middle-aged man, whom the same hacker is blackmailing over infidelity And if it’s code, why couldn’t it be a smart contract? Could harmful programs be launched onto blockchains and live there forever? Close your eyes and stretch your imagination towards 2030, when the world could play host to webs of code, data and physical devices that are immune to external regulation, empowered to carry to extremes the depravity or immorality of its creators. Here are some disturbing scenarios that become possible: - was a breakthrough in anonymous exchange, powered by Bitcoin. Its ultimate weakness (and one which led to its fall) was reliance on the postal system. In the future, mixed networks of drones and people, launched by anonymous groups and sustained by market dynamics, may provide programmable delivery services. With this type of infrastructure, Silk Road type marketplaces can be built using nothing but smart contracts. They could then deliver a whole supply chain of physical goods in a way that will be very hard to stop or trace, from UI to payments to fulfillment (and even financing to merchants). Just as Bittorrent gave Napster its wings, these networks will enable to proliferation of illicit markets like Silk Road Ungovernable Illicit Markets Silk Road - home addresses can be easily found today in leaked databases. Hackers could write a smart contract, lease an armed drone or robot and send it to a victim’s house. The device would threaten to harm life or property unless the victim scans a QR code and wires them a certain amount of money in cryptocurrency within a short deadline Blackmail - proceeds from illegal activities can be transformed from one cryptocurrency to another via decentralized exchanges with no KYC mechanisms (as far as I know, most decentralized exchanges today don’t do KYC). Privacy coins (ZCash) go straight for the jugular, where even if KYC requirements were met, it is near impossible to trace the provenance of each coin. Even anonymous cash-to-crypto exchange becomes possible, if hackers build an Ungovernable Illicit Market (as described above) in the spirit of and send a drone to pick up cash from you in a street corner Money Laundering Local Bitcoins - once enough identity and real-time biometric data is available on blockchains, we could see the emergence of assassination markets on smart contracts. Assassins can stake funds to communicate confidence in the outcome and proceed to carry out the crime. Once a public blockchain confirms that the victim’s heart is no longer beating (or that a video drone passed by and confirmed the outcome), a smart contract would reward the perpetrator Assassination Markets - it’s very easy to imagine how censorship-resistant file systems (Storj) can be combined with censorship-resistant VPN systems (Orchid) to produce, host and consume illegal content (insert your favorite genre of illicit content here) Unstoppable content networks - DAO’s can help groups of hackers “incorporate” as a DACO, exploiting the self-governing powers of DAOs to raise funds, communicate anonymously, vote, and launch any number of malicious schemes. Members would share the profits of those schemes in the long term Decentralized Autonomous Crime Organizations (DACO) Very much like malicious AI, blockchain viruses and DACOs can take a life of their own as long as benefits from having them and can sustain markets around them. What can we do to prevent the rise of viruses and DACOs in the blockchain economy? The answers aren’t simple, but here are a few thoughts: someone - blockchains such as Tezos are trying to raise awareness to the topic of governance, but both they and Ethereum haven’t passed the test of time when it comes to controversial or malicious contracts. Human moderation and voting are powerful ways to eliminate malicious content that’s been reported. Blockchain “anti-viruses” and mandatory code reviews can come later as the industry matures Built-in governance in blockchains - if the predetermined blockchain governance doesn’t eliminate certain bad content, communities can hard-fork the network into a cleaner version (just like Ethereum was forked into ETC after the attack). However, forks are extremely expensive to the ecosystem. Even worse, as long as (even 5% of the original ecosystem) will see the economic benefit in serving “blockchain viruses”, those viruses will be able to find infrastructure to run on Forks DAO someone - the examples above prove (again) that governments and blockchains are on a collision course in the long term, with blockchain money being the canary in the coalmine. As the case of Silk Road proved, proactive governments are the ultimate actor in securing our world. The key to preventing many (but not all) blockchain viruses is in controlling infrastructure, especially when it comes to physical devices and online-offline exchange of information. If countries want to prevent the rise of blockchain viruses and DACO’s, they should tightly control the flow of real-world information to blockchain code (e.g. biometric data, videos or transactions from the banking system). They may also want to control the exchange of crypto-to-fiat (centralized exchanges currently suffer problems with banks, but a compliant centralized exchange can be a government’s best friend in the long term). Finally, they should strive to decide what physical infrastructure can be offered via blockchains (should people be allowed to launch smart-contract controlled delivery networks, or smart-contract controlled 3D printing services?). Regulation & Enforcement is an interesting emerging weapon in traditional cybersecurity. Can we run networks code and devices (whether on blockchain or not) that would proactively detect, counter, and mislead bad actors? Who would launch and fund the operations of such networks? Defense by offense- active cyber defense Can the vision of unstoppable, open, market-driven networks turn into a nightmarish anarchy? Are law enforcers and skills for such a world? Future real world battlefields may not be about heroes with fast cars, gadgets or superpowers- but talented coders in a virtual cat-and-mouse game. Stay tuned. equipped with the right technology Thanks to my friends Michelle Lai , Thomas Glucksmann, Benjamin Joffe and Itamar Har Even for their thoughtful feedback & contributions

Stories, News & Mentions about COAVMI

Categories

Trending Topics

Stories, News & Mentions about COAVMI

Coavmi’s Stories on HackerNoon

Categories

Trending Topics