DevOps Skills

The broad picture. Skills to address the “from Code to Infrastructure” paradigm. Bridging ends from code producers to deployment in production – mindset of all involved, get a sense of the process as well do the automation of it and the orchestration and monitoring.

Collaborate with internal management teams involved in the DevOps process and stay familiar with the objectives, roadmap, blocking issues and other project areas.
Have the skills to mentor and advise team members on the best ways to deliver code, what tools to use when coding and how to test the latest features.

The target. Fast provisioning: be able to setup new machines fast. Good monitoring: to be quickly able to diagnose failures and trace them down. Quickly rollback to a previous version of the microservice. Rapid app deployment through fully automated pipelines. Create the Devops mindset / culture.

DevOps engineers need to know how to use and understand the roles of the following types of tools:
1. Version control: GitHub, GitLab
2. Continuous Integration servers: code coming in repository server and triggers build and doc: Jenkins, GitLab CI, Atlassian Bamboo, Circle CI, GitHub Actions
3. Configuration management: Software Configuration Management SCM Tools: Configuration management occurs when a configuration platform is used to automate, monitor, design and manage otherwise manual configuration processes. System-wide changes take place across servers and networks, storage, applications, and other managed systems: Puppet, Ansible, Chef
4. Deployment automation: Ansible Tower, Bamboo
5. Containers: containerd, Docker, Artifactory
6. Infrastructure Orchestration: automating the provisioning of the infrastructure services needed to support an app moving into production – in the right order, is orchestration: Terraform, Ansible (also Config. Management Tool), Chef, Kubernetes
7. Monitoring and analytics: Prometheus, Datadog, Splunk
8. Testing and Cloud Quality tools: a test automation platform uses scripts to automate the whole process of software testing. Identify the tests that need to be automated. Research and analyze the automation tools that meet your automation needs and budget. Based on the requirements, shortlist two most suitable tools. Do a pilot for two best tools and select the better one. Discuss the chosen automation tools with other stakeholders, explain the choice, and get their approval. Proceed to test automation
Tools: Kobiton, Eggplant, TestProject, LambdaTest
9. Network protocols from layers 4 to 7, nginx, caching, Service Mesh.
10. Programming skills with Java, Shell, Python, JS, Ruby…

Also:
Monitoring production environments
Performance measurements
Security
Cloud administration
Get proper alerts when something is wrong or unavailable
Help resolve problems either through online support or technical troubleshooting

How Tos – Know Hows

Some short and handy how tos.

How to configure Eclipse with PyDev, Python and Plotly (Graphs) on Windows 10?
You need to have all on the same bits. Windows 10 on 64bits then also Eclipse and Java.
You can get old Eclipse installations from: https://www.eclipse.org/downloads/packages/release
Get old Java installations (without login to java oracle sign-on): https://www.oracle.com/java/technologies/javase-jdk16-downloads.html and use right-click.
Add JAVAHOME to PATH. JAVAHOME=path to java bin.
Drag and drop https://marketplace.eclipse.org/content/pydev-python-ide-eclipse into Eclipse.
Install Python: https://www.python.org/downloads/ and add to PATH to Python and Python Scripts.
Install plotly, numby, pandas with the Python Scripts command: pip install plotly
Revise: Project -> Properties -> PYTHONPATH and Preferences -> PyDev -> Interpreters -> Python Interpreters (correct location and plotly, etc. versions).

How to recover your unprocessed Ethereum transaction due to too low fees allocated to it?
Transaction stays in the processing queue and they are processed in a “competitive” way, the more the fee the more likely some processing miners will take it. Sometimes the fee allocated to the transaction can be too low and no miner will process it anytime soon. What is this fee? This is the gas fee – the fee that miners earn for their job to process transactions in the blockchain.
In order to recover this transaction, to get (at almost no cost) all the value back to your wallet, you have to:
– check the transaction specifics with an eth explorer
– identify the status (processing/pending or similar) and note the nonce value of it (a number)
– initiate a new transaction to yourself: to your eth address, of smallest amount possible (0 if possible), assign the same nonce to this transaction and give to it a suggested “fast” high gas fee (it is a small fee for this second transaction though due to the 0 amount).
This way we instruct the transaction queue to cancel the original unprocessed transaction. Note that this only works for not yet processed in the blockchain transactions. Once a transaction is successful in the blockchain, it can’t be undone.

How to make the system notification tray area wider on Windows 10?
This can’t be made with normal settings. But you can make all the icons visible by the simple drag and drop action. Drag and drop the icons from the pop-up part of the tray to the visible tray area.

How to convert multiple mkv video files in a folder with a batch file on Windows 10?
I needed it with little quality loss, with re-scaling to 16:9 resolution, with small but ok mp3 audio and also with burned in subtitles (from the source mkv itself).
You need to install ffmpeg then copy ffmpeg.exe in the movies folder. Create a bat file there (make sure the destination folder exists) as in the script below:

FOR /F "tokens=*" %%G IN ('dir /b *.mkv') DO ffmpeg -i "%%G" -c:v libx264 -preset veryfast -crf 20 -c:a mp3 -af "aformat=sample_fmts=s16" -ar 44100 -b:a 160k -vf "scale=1280:720,setdar=16/9,subtitles='%%G' :si=0" -r 23.976 "e:\dest\%%G"

Frame rate should be on both source and destination files the same. Sometimes this information is misread by ffmpeg (eg. that the source is only 12 fps, thus the re-encoding will drop frames to fit that value). One way to see the real frame rate is using a software like MediaInfo and once determined the exact frame rate, it should be specified with the “-r” parameter.
In case the source subtitle is in vob format (not text) there is no need to extract it, but to use instead the “filter_complex” parameter (and discard the “-vf” part) to overlay the image subtitle:

FOR /F "tokens=*" %%G IN ('dir /b *.mkv') DO ffmpeg -threads 4 -i "%%G" -filter_complex "[0:v]scale=1280:720[video],[0:s:0]scale=1280:720[subs],[video][subs]overlay[out]" -map "[out]" -map 0:a:0 -c:v libx264 -preset superfast -crf 20 -c:a mp3 -af "aformat=sample_fmts=s16" -ar 44100 -b:a 160k -aspect 16:9 -s 1280x720 -r 23.976 "e:\dest\%%G"

How to delete the old Windows 10 folders that won’t delete with normal delete?
After making a fresh Windows 10 installation the old users and Windows.old folders can’t be removed with normal delete action in explorer, even as administrator. This happens mainly due to inherited credentials of owners of the old installation. To some extend the cleanmgr tool helps.
I use these steps and a bat file: rename the folder in order to make sure it is not locked (restart maybe needed), then use the script below to take ownership and give yourself rights to it, and then delete.

takeown /F "ProgramData_" /A /R /D Y
icacls "ProgramData_" /T /grant yourUserName:F
rd /s /q "ProgramData_"

How to see your different kind of cryptos and on different blockchain networks?
Use a multi wallet crypto wallet. A more general wallet that is able to see crypto tokens that are on different blockckain protocols. For ex. Ether is an ERC-20 token. ERC-20 is the standard for Ethereum network. Upon this standard all kinds of tokens exists (from other projects in the crypto/defi space).
Other blockchains provided their own standard protocol upon which one can create the crypto assets (tokens, contracts). For example ESDT standard for Elrond network, or BEP2 on Binance’s blockchain.
You can use the keys you generated your initial wallet to import them into the new multi wallet. This way you have access to all your different crypto assets.

How to create an MSDOS boot CD and update the BIOS from it?
Sometimes on older BIOS, while upgrading the CPU, you get the message: “Patch for installed CPU not found. Please run the BIOS flash update diskette”. While on Windows, a simple flash update/microcode update won’t work directly, most of flash tools need to run in “real mode”.
While still possible to burn an MSDOS iso image file on a CD, there is sometimes need to have access, while in DOS, to additional files from a floppy/FAT file system. Most of the systems do not have a working floppy anymore nor USB friendly, nor access to a FAT partition. So here are the steps to make a suitable MSDOS boot CD with the BIOS flash/patch utility on it:
– download MSDOS 6.22 iso from Microsoft.
– install WinImage (www.winimage.com) & Nero.
– in WinImage open the ISO and save it/export (boot sector property export) to an IMA file. Open IMA again and modify format to 2.88MB. Add/inject necessary flash files to it (total size < 2.88MB). Save IMA again: you have a new image file that besides DOS boot option you have also the microcode/patch on it.
– in Nero create a new Image Recorder project with the CD Boot compilation option. Create from image file: browse/add the IMA file at Boot tab. Expert settings: Floppy emulation 2.88MB. Now press New and then Burn (at Image Recorder): this will create the NRG image compatible with Nero. This image can then be burned on a CD-R/W. Attention: disk content may not be visible on Windows! But boot from it, should work and have all files visible within the MS DOS environment.

How to install and use the Creative Audigy 2 ZS Platinum audio PCI card on Windows7 64bit?
The Creative Audigy 2 ZS audio card was the first audio interface that was THX certified. For the first time home users could benefit from high audio digital standards on their PC. Card came in the Platinum Pro variant also, with the external module that can be placed on desktop allowing also more “pro” users to make use of various I/Os. One in all it was and still is a great audio interface, with strong features suitable for gamer, home theater setup, and…home musicians!
– DACs 24-bit/192kHz with ASIO 2.0 Low Latency Multi-Track Recording Support (24-bit/96kHz), 108dB SNR
– Dolby Digital up to 7.1 / THX, SPDIF, Remote Control
The problem nowadays is that the software and driver support for this PCI old card has been terminated for some time. The use on Windows7 and up is problematic, due to lack of software and drivers support it is difficult to install and use properly.
In order to still be able to use this card, especially on 64-bit platforms here some tips:
– install drivers from package SBA5_PCDRV_L11_3_01_0056B (install only the drivers for Windows 10 from this setup for Audigy 5) thus: unzip, search and right click and install the file: wdma_emu.inf
– install the Audigy 2 Refresh (try also these drivers, if needed) and software from: https://danielkawakami.blogspot.com
– install Audigy 2 package (drivers only if needed) of SBAX_PCDRV_LB_2_18_0017 from Creative support: https://support.creative.com/Downloads*
– install only the needed software from the original bundle Audigy 2 CD (with registering the dlls, if errors: C:\Windows\SysWOW64>regsvr32 ctscal.dll and ctthxcal.dll)
Following the steps above, on Windows7 64bit you should at least be able to nicely:
– have ASIO available for your DAW
– have a “What U Hear” on recording devices
– have the possibility to configure the audio interface from 2/2.1 and up with WDM\COMMON\CtPanel.exe
– have the possibility to play audio in 4.0 mode, with output 1 for Monitors and output 2 for Headphones (useful when not having/using the external module).
Please note that on Windows 7 x64 the original software is not working 100% (for ex. Surround Mixer or Calibration are buggy).
Please note that on Windows 10 x64 and up the drivers may not work at all for recording applications, only for playback. With “device could not be open” or “device not available” messages.

How disable updates on android/huawei phones?
Update is a necessary evil. There are situations when there is no possibility to perform the update. Windows is notorious about the handling of the updates. Nowadays also the phones are pushing these updates, more like Windows, totally ignoring: the space scarcity (ironically of the devices they made and certified for their system), the normal user settings.
There a silent war between the user and the update programs employed. For example after Windows 7, Microsoft created more than one service just for updates war. It is harder and harder for user to even attempt to stop them. On Huawei phones for example, it is useless to disable from the apps everything related to updates, after restarting, the settings are reverted and update messages still gets pushed as nothing happened.
Fortunately there is a Developer mode that one can enable on the huawei device. And from here there is a way to disable for good the update notifications. Here the steps for Huawei phones (no rooting!):
– Enable Developer mode by tapping more times the Build Number section on About at Settings.
– Enable USB Debugging from the Developer Options.
– Download and unzip ADB on your Desktop PC:
https://dl.google.com/android/repository/platform-tools-latest-windows.zip
For linux: https://dl.google.com/android/repository/platform-tools-latest-linux.zip
For older version use build numbers in link eg.: https://dl.google.com/android/repository/platform-tools_r27.0.0-windows.zip
– Connect Phone to the PC with the USB cable, in File Transfer mode.
– Within the folder where ADB is placed, you can use the following commands to disable the update:
adb shell devices (here you should see the connected device if everything OK)
adb shell pm list packages | grep huawei.android
adb shell pm disable-user com.huawei.android.hwouc

or
adb shell pm uninstall –user 0 com.huawei.android.hwouc
Note that the settings can be reactivated with the “enable” and “install” commands.
ADB: Android Debug Bridge (adb) is a versatile command-line tool that lets you communicate with a device. The adb command facilitates a variety of device actions, such as installing and debugging apps, and it provides access to a Unix shell that you can use to run a variety of commands on a device.

Job Breakthroughs

Startup vs. Larger Company:
Working for a smaller company is that you get to make more of an impact: Working in a larger corporation might have more benefits or a higher salary but a startup is where you can really make a difference and see the influence your work is having on the business. You’re heavily involved in each stage of production and your opinion is more likely to carry weight than at a larger, more structured operation. Decentralization of big companies would be done through tokenization. The shares will be done through ICOs.
Jobs in IT:
In Artificial Intelligence, the Internet of Things, data security, virtual reality and augmented reality, virtual worlds (and virtual assets) and bank-less, free nodes back-boned, Internet of payment. Jobs to see as or related to: big data engineer, Software 2.0 Engineer (maintain Neural Networks that write code), full-stack developer, security engineer, IoT architect and VR/AR engineer and hybrid engineers, with agile mindsets through the teams, with solid technology stacks knowledge that working together are able to bind different ends of the domain spectrum (similarly like DevOps is to the “from Code to Infrastructure” mindset paradigm), runners of decentralized Internet (sustained by Blockchain and other similar technologies yet to come, in order to back-up the Virtual Assets in the Virtual Worlds in the Decentralized Network).
Thus the skills needed to succeed in the IT jobs of tomorrow revolve around security certifications, programming and applications development, proficiency with cloud, decentralized architectures and mobile technologies, and other specialized skill sets giving also way to the hybrid IT roles that bind the business to IT.
Roles grow vertically based on business domain vs. technology stacks. For example: a Solutions Architect has the business domain knowledge but has also a technical background. He will develop complex technology solutions in a specific business domain. Software Architect knows in a deeper way the technology stacks. He will design the architecture of the technical implementation. Technical Lead is one with deeper knowledge of the, or a part of the technology stack. He designs using established patterns, coaches teams into the adopted technologies and unlocks teams in order to succeed in project delivery.
Data Scientists: it is essential for data scientists to work with languages like R, Python, SAS, Hadoop, Netezza in which they apply their knowledge in statistics, mathematics (algebra), matrices (multivariable) calculus. And to have a knowledge in platforms like MapReduce, GridGain, HPCC, Storm, Hive, Pig, Amazon S3.
The user as valuable “in the network” resource, in parallel digital universes (eg. Metaverse). Their actions should be monetized and generate income. We are producing valuable data even now by only navigating on FB, Google and other social networks which the system themselves uses it to become better (the long therm plan is building the future AI systems together). The “Internaut” will be one of the nicest job of the future.