62 skills found · Page 1 of 3
logangraham / ArXausalityA every-so-often-updated collection of every causality + machine learning paper submitted to arXiv in the recent past.
chrisneagu / FTC Skystone Dark Angels Romania 2020NOTICE This repository contains the public FTC SDK for the SKYSTONE (2019-2020) competition season. If you are looking for the current season's FTC SDK software, please visit the new and permanent home of the public FTC SDK: FtcRobotController repository Welcome! This GitHub repository contains the source code that is used to build an Android app to control a FIRST Tech Challenge competition robot. To use this SDK, download/clone the entire project to your local computer. Getting Started If you are new to robotics or new to the FIRST Tech Challenge, then you should consider reviewing the FTC Blocks Tutorial to get familiar with how to use the control system: FTC Blocks Online Tutorial Even if you are an advanced Java programmer, it is helpful to start with the FTC Blocks tutorial, and then migrate to the OnBot Java Tool or to Android Studio afterwards. Downloading the Project If you are an Android Studio programmer, there are several ways to download this repo. Note that if you use the Blocks or OnBot Java Tool to program your robot, then you do not need to download this repository. If you are a git user, you can clone the most current version of the repository: git clone https://github.com/FIRST-Tech-Challenge/SKYSTONE.git Or, if you prefer, you can use the "Download Zip" button available through the main repository page. Downloading the project as a .ZIP file will keep the size of the download manageable. You can also download the project folder (as a .zip or .tar.gz archive file) from the Downloads subsection of the Releases page for this repository. Once you have downloaded and uncompressed (if needed) your folder, you can use Android Studio to import the folder ("Import project (Eclipse ADT, Gradle, etc.)"). Getting Help User Documentation and Tutorials FIRST maintains online documentation with information and tutorials on how to use the FIRST Tech Challenge software and robot control system. You can access this documentation using the following link: SKYSTONE Online Documentation Note that the online documentation is an "evergreen" document that is constantly being updated and edited. It contains the most current information about the FIRST Tech Challenge software and control system. Javadoc Reference Material The Javadoc reference documentation for the FTC SDK is now available online. Click on the following link to view the FTC SDK Javadoc documentation as a live website: FTC Javadoc Documentation Documentation for the FTC SDK is also included with this repository. There is a subfolder called "doc" which contains several subfolders: The folder "apk" contains the .apk files for the FTC Driver Station and FTC Robot Controller apps. The folder "javadoc" contains the JavaDoc user documentation for the FTC SDK. Online User Forum For technical questions regarding the Control System or the FTC SDK, please visit the FTC Technology forum: FTC Technology Forum Release Information Version 5.5 (20200824-090813) Version 5.5 requires Android Studio 4.0 or later. New features Adds support for calling custom Java classes from Blocks OpModes (fixes SkyStone issue #161). Classes must be in the org.firstinspires.ftc.teamcode package. Methods must be public static and have no more than 21 parameters. Parameters declared as OpMode, LinearOpMode, Telemetry, and HardwareMap are supported and the argument is provided automatically, regardless of the order of the parameters. On the block, the sockets for those parameters are automatically filled in. Parameters declared as char or java.lang.Character will accept any block that returns text and will only use the first character in the text. Parameters declared as boolean or java.lang.Boolean will accept any block that returns boolean. Parameters declared as byte, java.lang.Byte, short, java.lang.Short, int, java.lang.Integer, long, or java.lang.Long, will accept any block that returns a number and will round that value to the nearest whole number. Parameters declared as float, java.lang.Float, double, java.lang.Double will accept any block that returns a number. Adds telemetry API method for setting display format Classic Monospace HTML (certain tags only) Adds blocks support for switching cameras. Adds Blocks support for TensorFlow Object Detection with a custom model. Adds support for uploading a custom TensorFlow Object Detection model in the Manage page, which is especially useful for Blocks and OnBotJava users. Shows new Control Hub blink codes when the WiFi band is switched using the Control Hub's button (only possible on Control Hub OS 1.1.2) Adds new warnings which can be disabled in the Advanced RC Settings Mismatched app versions warning Unnecessary 2.4 GHz WiFi usage warning REV Hub is running outdated firmware (older than version 1.8.2) Adds support for Sony PS4 gamepad, and reworks how gamepads work on the Driver Station Removes preference which sets gamepad type based on driver position. Replaced with menu which allows specifying type for gamepads with unknown VID and PID Attempts to auto-detect gamepad type based on USB VID and PID If gamepad VID and PID is not known, use type specified by user for that VID and PID If gamepad VID and PID is not known AND the user has not specified a type for that VID and PID, an educated guess is made about how to map the gamepad Driver Station will now attempt to automatically recover from a gamepad disconnecting, and re-assign it to the position it was assigned to when it dropped If only one gamepad is assigned and it drops: it can be recovered If two gamepads are assigned, and have different VID/PID signatures, and only one drops: it will be recovered If two gamepads are assigned, and have different VID/PID signatures, and BOTH drop: both will be recovered If two gamepads are assigned, and have the same VID/PID signatures, and only one drops: it will be recovered If two gamepads are assigned, and have the same VID/PID signatures, and BOTH drop: neither will be recovered, because of the ambiguity of the gamepads when they re-appear on the USB bus. There is currently one known edge case: if there are two gamepads with the same VID/PID signature plugged in, but only one is assigned, and they BOTH drop, it's a 50-50 chance of which one will be chosen for automatic recovery to the assigned position: it is determined by whichever one is re-enumerated first by the USB bus controller. Adds landscape user interface to Driver Station New feature: practice timer with audio cues New feature (Control Hub only): wireless network connection strength indicator (0-5 bars) New feature (Control Hub only): tapping on the ping/channel display will switch to an alternate display showing radio RX dBm and link speed (tap again to switch back) The layout will NOT autorotate. You can switch the layout from the Driver Station's settings menu. Breaking changes Removes support for Android versions 4.4 through 5.1 (KitKat and Lollipop). The minSdkVersion is now 23. Removes the deprecated LinearOpMode methods waitOneFullHardwareCycle() and waitForNextHardwareCycle() Enhancements Handles RS485 address of Control Hub automatically The Control Hub is automatically given a reserved address Existing configuration files will continue to work All addresses in the range of 1-10 are still available for Expansion Hubs The Control Hub light will now normally be solid green, without blinking to indicate the address The Control Hub will not be shown on the Expansion Hub Address Change settings page Improves REV Hub firmware updater The user can now choose between all available firmware update files Version 1.8.2 of the REV Hub firmware is bundled into the Robot Controller app. Text was added to clarify that Expansion Hubs can only be updated via USB. Firmware update speed was reduced to improve reliability Allows REV Hub firmware to be updated directly from the Manage webpage Improves log viewer on Robot Controller Horizontal scrolling support (no longer word wrapped) Supports pinch-to-zoom Uses a monospaced font Error messages are highlighted New color scheme Attempts to force-stop a runaway/stuck OpMode without restarting the entire app Not all types of runaway conditions are stoppable, but if the user code attempts to talk to hardware during the runaway, the system should be able to capture it. Makes various tweaks to the Self Inspect screen Renames "OS version" entry to "Android version" Renames "WiFi Direct Name" to "WiFi Name" Adds Control Hub OS version, when viewing the report of a Control Hub Hides the airplane mode entry, when viewing the report of a Control Hub Removes check for ZTE Speed Channel Changer Shows firmware version for all Expansion and Control Hubs Reworks network settings portion of Manage page All network settings are now applied with a single click The WiFi Direct channel of phone-based Robot Controllers can now be changed from the Manage page WiFi channels are filtered by band (2.4 vs 5 GHz) and whether they overlap with other channels The current WiFi channel is pre-selected on phone-based Robot Controllers, and Control Hubs running OS 1.1.2 or later. On Control Hubs running OS 1.1.2 or later, you can choose to have the system automatically select a channel on the 5 GHz band Improves OnBotJava New light and dark themes replace the old themes (chaos, github, chrome,...) the new default theme is light and will be used when you first update to this version OnBotJava now has a tabbed editor Read-only offline mode Improves function of "exit" menu item on Robot Controller and Driver Station Now guaranteed to be fully stopped and unloaded from memory Shows a warning message if a LinearOpMode exists prematurely due to failure to monitor for the start condition Improves error message shown when the Driver Station and Robot Controller are incompatible with each other Driver Station OpMode Control Panel now disabled while a Restart Robot is in progress Disables advanced settings related to WiFi direct when the Robot Controller is a Control Hub. Tint phone battery icons on Driver Station when low/critical. Uses names "Control Hub Portal" and "Control Hub" (when appropriate) in new configuration files Improve I2C read performance Very large improvement on Control Hub; up to ~2x faster with small (e.g. 6 byte) reads Not as apparent on Expansion Hubs connected to a phone Update/refresh build infrastructure Update to 'androidx' support library from 'com.android.support:appcompat', which is end-of-life Update targetSdkVersion and compileSdkVersion to 28 Update Android Studio's Android plugin to latest Fix reported build timestamp in 'About' screen Add sample illustrating manual webcam use: ConceptWebcam Bug fixes Fixes SkyStone issue #248 Fixes SkyStone issue #232 and modifies bulk caching semantics to allow for cache-preserving MANUAL/AUTO transitions. Improves performance when REV 2M distance sensor is unplugged Improves readability of Toast messages on certain devices Allows a Driver Station to connect to a Robot Controller after another has disconnected Improves generation of fake serial numbers for UVC cameras which do not provide a real serial number Previously some devices would assign such cameras a serial of 0:0 and fail to open and start streaming Fixes ftc_app issue #638. Fixes a slew of bugs with the Vuforia camera monitor including: Fixes bug where preview could be displayed with a wonky aspect ratio Fixes bug where preview could be cut off in landscape Fixes bug where preview got totally messed up when rotating phone Fixes bug where crosshair could drift off target when using webcams Fixes issue in UVC driver on some devices (ftc_app 681) if streaming was started/stopped multiple times in a row Issue manifested as kernel panic on devices which do not have this kernel patch. On affected devices which do have the patch, the issue was manifest as simply a failure to start streaming. The Tech Team believes that the root cause of the issue is a bug in the Linux kernel XHCI driver. A workaround was implemented in the SDK UVC driver. Fixes bug in UVC driver where often half the frames from the camera would be dropped (e.g. only 15FPS delivered during a streaming session configured for 30FPS). Fixes issue where TensorFlow Object Detection would show results whose confidence was lower than the minimum confidence parameter. Fixes a potential exploitation issue of CVE-2019-11358 in OnBotJava Fixes changing the address of an Expansion Hub with additional Expansion Hubs connected to it Preserves the Control Hub's network connection when "Restart Robot" is selected Fixes issue where device scans would fail while the Robot was restarting Fix RenderScript usage Use androidx.renderscript variant: increased compatibility Use RenderScript in Java mode, not native: simplifies build Fixes webcam-frame-to-bitmap conversion problem: alpha channel wasn't being initialized, only R, G, & B Fixes possible arithmetic overflow in Deadline Fixes deadlock in Vuforia webcam support which could cause 5-second delays when stopping OpMode Version 5.4 (20200108-101156) Fixes SkyStone issue #88 Adds an inspection item that notes when a robot controller (Control Hub) is using the factory default password. Fixes SkyStone issue #61 Fixes SkyStone issue #142 Fixes ftc_app issue #417 by adding more current and voltage monitoring capabilities for REV Hubs. Fixes a crash sometimes caused by OnBotJava activity Improves OnBotJava autosave functionality ftc_app #738 Fixes system responsiveness issue when an Expansion Hub is disconnected Fixes issue where IMU initialization could prevent Op Modes from stopping Fixes issue where AndroidTextToSpeech.speak() would fail if it was called too early Adds telemetry.speak() methods and blocks, which cause the Driver Station (if also updated) to speak text Adds and improves Expansion Hub-related warnings Improves Expansion Hub low battery warning Displays the warning immediately after the hub reports it Specifies whether the condition is current or occurred temporarily during an OpMode run Displays which hubs reported low battery Displays warning when hub loses and regains power during an OpMode run Fixes the hub's LED pattern after this condition Displays warning when Expansion Hub is not responding to commands Specifies whether the condition is current or occurred temporarily during an OpMode run Clarifies warning when Expansion Hub is not present at startup Specifies that this condition requires a Robot Restart before the hub can be used. The hub light will now accurately reflect this state Improves logging and reduces log spam during these conditions Syncs the Control Hub time and timezone to a connected web browser programming the robot, if a Driver Station is not available. Adds bulk read functionality for REV Hubs A bulk caching mode must be set at the Hub level with LynxModule#setBulkCachingMode(). This applies to all relevant SDK hardware classes that reference that Hub. The following following Hub bulk caching modes are available: BulkCachingMode.OFF (default): All hardware calls operate as usual. Bulk data can read through LynxModule#getBulkData() and processed manually. BulkCachingMode.AUTO: Applicable hardware calls are served from a bulk read cache that is cleared/refreshed automatically to ensure identical commands don't hit the same cache. The cache can also be cleared manually with LynxModule#clearBulkCache(), although this is not recommended. (advanced users) BulkCachingMode.MANUAL: Same as BulkCachingMode.AUTO except the cache is never cleared automatically. To avoid getting stale data, the cache must be manually cleared at the beginning of each loop body or as the user deems appropriate. Removes PIDF Annotation values added in Rev 5.3 (to AndyMark, goBILDA and TETRIX motor configurations). The new motor types will still be available but their Default control behavior will revert back to Rev 5.2 Adds new ConceptMotorBulkRead sample Opmode to demonstrate and compare Motor Bulk-Read modes for reducing I/O latencies. Version 5.3 (20191004-112306) Fixes external USB/UVC webcam support Makes various bugfixes and improvements to Blocks page, including but not limited to: Many visual tweaks Browser zoom and window resize behave better Resizing the Java preview pane works better and more consistently across browsers The Java preview pane consistently gets scrollbars when needed The Java preview pane is hidden by default on phones Internet Explorer 11 should work Large dropdown lists display properly on lower res screens Disabled buttons are now visually identifiable as disabled A warning is shown if a user selects a TFOD sample, but their device is not compatible Warning messages in a Blocks op mode are now visible by default. Adds goBILDA 5201 and 5202 motors to Robot Configurator Adds PIDF Annotation values to AndyMark, goBILDA and TETRIX motor configurations. This has the effect of causing the RUN_USING_ENCODERS and RUN_TO_POSITION modes to use PIDF vs PID closed loop control on these motors. This should provide more responsive, yet stable, speed control. PIDF adds Feedforward control to the basic PID control loop. Feedforward is useful when controlling a motor's speed because it "anticipates" how much the control voltage must change to achieve a new speed set-point, rather than requiring the integrated error to change sufficiently. The PIDF values were chosen to provide responsive, yet stable, speed control on a lightly loaded motor. The more heavily a motor is loaded (drag or friction), the more noticable the PIDF improvement will be. Fixes startup crash on Android 10 Fixes ftc_app issue #712 (thanks to FROGbots-4634) Fixes ftc_app issue #542 Allows "A" and lowercase letters when naming device through RC and DS apps. Version 5.2 (20190905-083277) Fixes extra-wide margins on settings activities, and placement of the new configuration button Adds Skystone Vuforia image target data. Includes sample Skystone Vuforia Navigation op modes (Java). Includes sample Skystone Vuforia Navigation op modes (Blocks). Adds TensorFlow inference model (.tflite) for Skystone game elements. Includes sample Skystone TensorFlow op modes (Java). Includes sample Skystone TensorFlow op modes (Blocks). Removes older (season-specific) sample op modes. Includes 64-bit support (to comply with Google Play requirements). Protects against Stuck OpModes when a Restart Robot is requested. (Thanks to FROGbots-4634) (ftc_app issue #709) Blocks related changes: Fixes bug with blocks generated code when hardware device name is a java or javascript reserved word. Shows generated java code for blocks, even when hardware items are missing from the active configuration. Displays warning icon when outdated Vuforia and TensorFlow blocks are used (SkyStone issue #27) Version 5.1 (20190820-222104) Defines default PIDF parameters for the following motors: REV Core Hex Motor REV 20:1 HD Hex Motor REV 40:1 HD Hex Motor Adds back button when running on a device without a system back button (such as a Control Hub) Allows a REV Control Hub to update the firmware on a REV Expansion Hub via USB Fixes SkyStone issue #9 Fixes ftc_app issue #715 Prevents extra DS User clicks by filtering based on current state. Prevents incorrect DS UI state changes when receiving new OpMode list from RC Adds support for REV Color Sensor V3 Adds a manual-refresh DS Camera Stream for remotely viewing RC camera frames. To show the stream on the DS, initialize but do not run a stream-enabled opmode, select the Camera Stream option in the DS menu, and tap the image to refresh. This feature is automatically enabled when using Vuforia or TFOD—no additional RC configuration is required for typical use cases. To hide the stream, select the same menu item again. Note that gamepads are disabled and the selected opmode cannot be started while the stream is open as a safety precaution. To use custom streams, consult the API docs for CameraStreamServer#setSource and CameraStreamSource. Adds many Star Wars sounds to RobotController resources. Added SKYSTONE Sounds Chooser Sample Program. Switches out startup, connect chimes, and error/warning sounds for Star Wars sounds Updates OnBot Java to use a WebSocket for communication with the robot The OnBot Java page no longer has to do a full refresh when a user switches from editing one file to another Known issues: Camera Stream The Vuforia camera stream inherits the issues present in the phone preview (namely ftc_app issue #574). This problem does not affect the TFOD camera stream even though it receives frames from Vuforia. The orientation of the stream frames may not always match the phone preview. For now, these frames may be rotated manually via a custom CameraStreamSource if desired. OnBotJava Browser back button may not always work correctly It's possible for a build to be queued, but not started. The OnBot Java build console will display a warning if this occurs. A user might not realize they are editing a different file if the user inadvertently switches from one file to another since this switch is now seamless. The name of the currently open file is displayed in the browser tab. Version 5.0 (built on 19.06.14) Support for the REV Robotics Control Hub. Adds a Java preview pane to the Blocks editor. Adds a new offline export feature to the Blocks editor. Display wifi channel in Network circle on Driver Station. Adds calibration for Logitech C270 Updates build tooling and target SDK. Compliance with Google's permissions infrastructure (Required after build tooling update). Keep Alives to mitigate the Motorola wifi scanning problem. Telemetry substitute no longer necessary. Improves Vuforia error reporting. Fixes ftctechnh/ftc_app issues 621, 713. Miscellaneous bug fixes and improvements. Version 4.3 (built on 18.10.31) Includes missing TensorFlow-related libraries and files. Version 4.2 (built on 18.10.30) Includes fix to avoid deadlock situation with WatchdogMonitor which could result in USB communication errors. Comm error appeared to require that user disconnect USB cable and restart the Robot Controller app to recover. robotControllerLog.txt would have error messages that included the words "E RobotCore: lynx xmit lock: #### abandoning lock:" Includes fix to correctly list the parent module address for a REV Robotics Expansion Hub in a configuration (.xml) file. Bug in versions 4.0 and 4.1 would incorrect list the address module for a parent REV Robotics device as "1". If the parent module had a higher address value than the daisy-chained module, then this bug would prevent the Robot Controller from communicating with the downstream Expansion Hub. Added requirement for ACCESS_COARSE_LOCATION to allow a Driver Station running Android Oreo to scan for Wi-Fi Direct devices. Added google() repo to build.gradle because aapt2 must be downloaded from the google() repository beginning with version 3.2 of the Android Gradle Plugin. Important Note: Android Studio users will need to be connected to the Internet the first time build the ftc_app project. Internet connectivity is required for the first build so the appropriate files can be downloaded from the Google repository. Users should not need to be connected to the Internet for subsequent builds. This should also fix buid issue where Android Studio would complain that it "Could not find com.android.tools.lint:lint-gradle:26.1.4" (or similar). Added support for REV Spark Mini motor controller as part of the configuration menu for a servo/PWM port on the REV Expansion Hub. Provide examples for playing audio files in an Op Mode. Block Development Tool Changes Includes a fix for a problem with the Velocity blocks that were reported in the FTC Technology forum (Blocks Programming subforum). Change the "Save completed successfully." message to a white color so it will contrast with a green background. Fixed the "Download image" feature so it will work if there are text blocks in the op mode. Introduce support for Google's TensorFlow Lite technology for object detetion for 2018-2019 game. TensorFlow lite can recognize Gold Mineral and Silver Mineral from 2018-2019 game. Example Java and Block op modes are included to show how to determine the relative position of the gold block (left, center, right). Version 4.1 (released on 18.09.24) Changes include: Fix to prevent crash when deprecated configuration annotations are used. Change to allow FTC Robot Controller APK to be auto-updated using FIRST Global Control Hub update scripts. Removed samples for non supported / non legal hardware. Improvements to Telemetry.addData block with "text" socket. Updated Blocks sample op mode list to include Rover Ruckus Vuforia example. Update SDK library version number. Version 4.0 (released on 18.09.12) Changes include: Initial support for UVC compatible cameras If UVC camera has a unique serial number, RC will detect and enumerate by serial number. If UVC camera lacks a unique serial number, RC will only support one camera of that type connected. Calibration settings for a few cameras are included (see TeamCode/src/main/res/xml/teamwebcamcalibrations.xml for details). User can upload calibration files from Program and Manage web interface. UVC cameras seem to draw a fair amount of electrical current from the USB bus. This does not appear to present any problems for the REV Robotics Control Hub. This does seem to create stability problems when using some cameras with an Android phone-based Robot Controller. FTC Tech Team is investigating options to mitigate this issue with the phone-based Robot Controllers. Updated sample Vuforia Navigation and VuMark Op Modes to demonstrate how to use an internal phone-based camera and an external UVC webcam. Support for improved motor control. REV Robotics Expansion Hub firmware 1.8 and greater will support a feed forward mechanism for closed loop motor control. FTC SDK has been modified to support PIDF coefficients (proportional, integral, derivative, and feed forward). FTC Blocks development tool modified to include PIDF programming blocks. Deprecated older PID-related methods and variables. REV's 1.8.x PIDF-related changes provide a more linear and accurate way to control a motor. Wireless Added 5GHz support for wireless channel changing for those devices that support it. Tested with Moto G5 and E4 phones. Also tested with other (currently non-approved) phones such as Samsung Galaxy S8. Improved Expansion Hub firmware update support in Robot Controller app Changes to make the system more robust during the firmware update process (when performed through Robot Controller app). User no longer has to disconnect a downstream daisy-chained Expansion Hub when updating an Expansion Hub's firmware. If user is updating an Expansion Hub's firmware through a USB connection, he/she does not have to disconnect RS485 connection to other Expansion Hubs. The user still must use a USB connection to update an Expansion Hub's firmware. The user cannot update the Expansion Hub firmware for a downstream device that is daisy chained through an RS485 connection. If an Expansion Hub accidentally gets "bricked" the Robot Controller app is now more likely to recognize the Hub when it scans the USB bus. Robot Controller app should be able to detect an Expansion Hub, even if it accidentally was bricked in a previous update attempt. Robot Controller app should be able to install the firmware onto the Hub, even if if accidentally was bricked in a previous update attempt. Resiliency FTC software can detect and enable an FTDI reset feature that is available with REV Robotics v1.8 Expansion Hub firmware and greater. When enabled, the Expansion Hub can detect if it hasn't communicated with the Robot Controller over the FTDI (USB) connection. If the Hub hasn't heard from the Robot Controller in a while, it will reset the FTDI connection. This action helps system recover from some ESD-induced disruptions. Various fixes to improve reliability of FTC software. Blocks Fixed errors with string and list indices in blocks export to java. Support for USB connected UVC webcams. Refactored optimized Blocks Vuforia code to support Rover Ruckus image targets. Added programming blocks to support PIDF (proportional, integral, derivative and feed forward) motor control. Added formatting options (under Telemetry and Miscellaneous categories) so user can set how many decimal places to display a numerical value. Support to play audio files (which are uploaded through Blocks web interface) on Driver Station in addition to the Robot Controller. Fixed bug with Download Image of Blocks feature. Support for REV Robotics Blinkin LED Controller. Support for REV Robotics 2m Distance Sensor. Added support for a REV Touch Sensor (no longer have to configure as a generic digital device). Added blocks for DcMotorEx methods. These are enhanced methods that you can use when supported by the motor controller hardware. The REV Robotics Expansion Hub supports these enhanced methods. Enhanced methods include methods to get/set motor velocity (in encoder pulses per second), get/set PIDF coefficients, etc.. Modest Improvements in Logging Decrease frequency of battery checker voltage statements. Removed non-FTC related log statements (wherever possible). Introduced a "Match Logging" feature. Under "Settings" a user can enable/disable this feature (it's disabled by default). If enabled, user provides a "Match Number" through the Driver Station user interface (top of the screen). The Match Number is used to create a log file specifically with log statements from that particular Op Mode run. Match log files are stored in /sdcard/FIRST/matlogs on the Robot Controller. Once an op mode run is complete, the Match Number is cleared. This is a convenient way to create a separate match log with statements only related to a specific op mode run. New Devices Support for REV Robotics Blinkin LED Controller. Support for REV Robotics 2m Distance Sensor. Added configuration option for REV 20:1 HD Hex Motor. Added support for a REV Touch Sensor (no longer have to configure as a generic digital device). Miscellaneous Fixed some errors in the definitions for acceleration and velocity in our javadoc documentation. Added ability to play audio files on Driver Station When user is configuring an Expansion Hub, the LED on the Expansion Hub will change blink pattern (purple-cyan) to indicate which Hub is currently being configured. Renamed I2cSensorType to I2cDeviceType. Added an external sample Op Mode that demonstrates localization using 2018-2019 (Rover Ruckus presented by QualComm) Vuforia targets. Added an external sample Op Mode that demonstrates how to use the REV Robotics 2m Laser Distance Sensor. Added an external sample Op Mode that demonstrates how to use the REV Robotics Blinkin LED Controller. Re-categorized external Java sample Op Modes to "TeleOp" instead of "Autonomous". Known issues: Initial support for UVC compatible cameras UVC cameras seem to draw significant amount of current from the USB bus. This does not appear to present any problems for the REV Robotics Control Hub. This does seem to create stability problems when using some cameras with an Android phone-based Robot Controller. FTC Tech Team is investigating options to mitigate this issue with the phone-based Robot Controllers. There might be a possible deadlock which causes the RC to become unresponsive when using a UVC webcam with a Nougat Android Robot Controller. Wireless When user selects a wireless channel, this channel does not necessarily persist if the phone is power cycled. Tech Team is hoping to eventually address this issue in a future release. Issue has been present since apps were introduced (i.e., it is not new with the v4.0 release). Wireless channel is not currently displayed for WiFi Direct connections. Miscellaneous The blink indication feature that shows which Expansion Hub is currently being configured does not work for a newly created configuration file. User has to first save a newly created configuration file and then close and re-edit the file in order for blink indicator to work. Version 3.6 (built on 17.12.18) Changes include: Blocks Changes Uses updated Google Blockly software to allow users to edit their op modes on Apple iOS devices (including iPad and iPhone). Improvement in Blocks tool to handle corrupt op mode files. Autonomous op modes should no longer get switched back to tele-op after re-opening them to be edited. The system can now detect type mismatches during runtime and alert the user with a message on the Driver Station. Updated javadoc documentation for setPower() method to reflect correct range of values (-1 to +1). Modified VuforiaLocalizerImpl to allow for user rendering of frames Added a user-overrideable onRenderFrame() method which gets called by the class's renderFrame() method. Version 3.5 (built on 17.10.30) Changes with version 3.5 include: Introduced a fix to prevent random op mode stops, which can occur after the Robot Controller app has been paused and then resumed (for example, when a user temporarily turns off the display of the Robot Controller phone, and then turns the screen back on). Introduced a fix to prevent random op mode stops, which were previously caused by random peer disconnect events on the Driver Station. Fixes issue where log files would be closed on pause of the RC or DS, but not re-opened upon resume. Fixes issue with battery handler (voltage) start/stop race. Fixes issue where Android Studio generated op modes would disappear from available list in certain situations. Fixes problem where OnBot Java would not build on REV Robotics Control Hub. Fixes problem where OnBot Java would not build if the date and time on the Robot Controller device was "rewound" (set to an earlier date/time). Improved error message on OnBot Java that occurs when renaming a file fails. Removed unneeded resources from android.jar binaries used by OnBot Java to reduce final size of Robot Controller app. Added MR_ANALOG_TOUCH_SENSOR block to Blocks Programming Tool. Version 3.4 (built on 17.09.06) Changes with version 3.4 include: Added telemetry.update() statement for BlankLinearOpMode template. Renamed sample Block op modes to be more consistent with Java samples. Added some additional sample Block op modes. Reworded OnBot Java readme slightly. Version 3.3 (built on 17.09.04) This version of the software includes improves for the FTC Blocks Programming Tool and the OnBot Java Programming Tool. Changes with verion 3.3 include: Android Studio ftc_app project has been updated to use Gradle Plugin 2.3.3. Android Studio ftc_app project is already using gradle 3.5 distribution. Robot Controller log has been renamed to /sdcard/RobotControllerLog.txt (note that this change was actually introduced w/ v3.2). Improvements in I2C reliability. Optimized I2C read for REV Expansion Hub, with v1.7 firmware or greater. Updated all external/samples (available through OnBot and in Android project folder). Vuforia Added support for VuMarks that will be used for the 2017-2018 season game. Blocks Update to latest Google Blockly release. Sample op modes can be selected as a template when creating new op mode. Fixed bug where the blocks would disappear temporarily when mouse button is held down. Added blocks for Range.clip and Range.scale. User can now disable/enable Block op modes. Fix to prevent occasional Blocks deadlock. OnBot Java Significant improvements with autocomplete function for OnBot Java editor. Sample op modes can be selected as a template when creating new op mode. Fixes and changes to complete hardware setup feature. Updated (and more useful) onBot welcome message. Known issues: Android Studio After updating to the new v3.3 Android Studio project folder, if you get error messages indicating "InvalidVirtualFileAccessException" then you might need to do a File->Invalidate Caches / Restart to clear the error. OnBot Java Sometimes when you push the build button to build all op modes, the RC returns an error message that the build failed. If you press the build button a second time, the build typically suceeds. Version 3.2 (built on 17.08.02) This version of the software introduces the "OnBot Java" Development Tool. Similar to the FTC Blocks Development Tool, the FTC OnBot Java Development Tool allows a user to create, edit and build op modes dynamically using only a Javascript-enabled web browser. The OnBot Java Development Tool is an integrated development environment (IDE) that is served up by the Robot Controller. Op modes are created and edited using a Javascript-enabled browser (Google Chromse is recommended). Op modes are saved on the Robot Controller Android device directly. The OnBot Java Development Tool provides a Java programming environment that does NOT need Android Studio. Changes with version 3.2 include: Enhanced web-based development tools Introduction of OnBot Java Development Tool. Web-based programming and management features are "always on" (user no longer needs to put Robot Controller into programming mode). Web-based management interface (where user can change Robot Controller name and also easily download Robot Controller log file). OnBot Java, Blocks and Management features available from web based interface. Blocks Programming Development Tool: Changed "LynxI2cColorRangeSensor" block to "REV Color/range sensor" block. Fixed tooltip for ColorSensor.isLightOn block. Added blocks for ColorSensor.getNormalizedColors and LynxI2cColorRangeSensor.getNormalizedColors. Added example op modes for digital touch sensor and REV Robotics Color Distance sensor. User selectable color themes. Includes many minor enhancements and fixes (too numerous to list). Known issues: Auto complete function is incomplete and does not support the following (for now): Access via this keyword Access via super keyword Members of the super cloass, not overridden by the class Any methods provided in the current class Inner classes Can't handle casted objects Any objects coming from an parenthetically enclosed expression Version 3.10 (built on 17.05.09) This version of the software provides support for the REV Robotics Expansion Hub. This version also includes improvements in the USB communication layer in an effort to enhance system resiliency. If you were using a 2.x version of the software previously, updating to version 3.1 requires that you also update your Driver Station software in addition to updating the Robot Controller software. Also note that in version 3.10 software, the setMaxSpeed and getMaxSpeed methods are no longer available (not deprecated, they have been removed from the SDK). Also note that the the new 3.x software incorporates motor profiles that a user can select as he/she configures the robot. Changes include: Blocks changes Added VuforiaTrackableDefaultListener.getPose and Vuforia.trackPose blocks. Added optimized blocks support for Vuforia extended tracking. Added atan2 block to the math category. Added useCompetitionFieldTargetLocations parameter to Vuforia.initialize block. If set to false, the target locations are placed at (0,0,0) with target orientation as specified in https://github.com/gearsincorg/FTCVuforiaDemo/blob/master/Robot_Navigation.java tutorial op mode. Incorporates additional improvements to USB comm layer to improve system resiliency (to recover from a greater number of communication disruptions). Additional Notes Regarding Version 3.00 (built on 17.04.13) In addition to the release changes listed below (see section labeled "Version 3.00 (built on 17.04.013)"), version 3.00 has the following important changes: Version 3.00 software uses a new version of the FTC Robocol (robot protocol). If you upgrade to v3.0 on the Robot Controller and/or Android Studio side, you must also upgrade the Driver Station software to match the new Robocol. Version 3.00 software removes the setMaxSpeed and getMaxSpeed methods from the DcMotor class. If you have an op mode that formerly used these methods, you will need to remove the references/calls to these methods. Instead, v3.0 provides the max speed information through the use of motor profiles that are selected by the user during robot configuration. Version 3.00 software currently does not have a mechanism to disable extra i2c sensors. We hope to re-introduce this function with a release in the near future. Version 3.00 (built on 17.04.13) *** Use this version of the software at YOUR OWN RISK!!! *** This software is being released as an "alpha" version. Use this version at your own risk! This pre-release software contains SIGNIFICANT changes, including changes to the Wi-Fi Direct pairing mechanism, rewrites of the I2C sensor classes, changes to the USB/FTDI layer, and the introduction of support for the REV Robotics Expansion Hub and the REV Robotics color-range-light sensor. These changes were implemented to improve the reliability and resiliency of the FTC control system. Please note, however, that version 3.00 is considered "alpha" code. This code is being released so that the FIRST community will have an opportunity to test the new REV Expansion Hub electronics module when it becomes available in May. The developers do not recommend using this code for critical applications (i.e., competition use). *** Use this version of the software at YOUR OWN RISK!!! *** Changes include: Major rework of sensor-related infrastructure. Includes rewriting sensor classes to implement synchronous I2C communication. Fix to reset Autonomous timer back to 30 seconds. Implementation of specific motor profiles for approved 12V motors (includes Tetrix, AndyMark, Matrix and REV models). Modest improvements to enhance Wi-Fi P2P pairing. Fixes telemetry log addition race. Publishes all the sources (not just a select few). Includes Block programming improvements Addition of optimized Vuforia blocks. Auto scrollbar to projects and sounds pages. Fixed blocks paste bug. Blocks execute after while-opModeIsActive loop (to allow for cleanup before exiting op mode). Added gyro integratedZValue block. Fixes bug with projects page for Firefox browser. Added IsSpeaking block to AndroidTextToSpeech. Implements support for the REV Robotics Expansion Hub Implements support for integral REV IMU (physically installed on I2C bus 0, uses same Bosch BNO055 9 axis absolute orientation sensor as Adafruit 9DOF abs orientation sensor). - Implements support for REV color/range/light sensor. Provides support to update Expansion Hub firmware through FTC SDK. Detects REV firmware version and records in log file. Includes support for REV Control Hub (note that the REV Control Hub is not yet approved for FTC use). Implements FTC Blocks programming support for REV Expansion Hub and sensor hardware. Detects and alerts when I2C device disconnect. Version 2.62 (built on 17.01.07) Added null pointer check before calling modeToByte() in finishModeSwitchIfNecessary method for ModernRoboticsUsbDcMotorController class. Changes to enhance Modern Robotics USB protocol robustness. Version 2.61 (released on 16.12.19) Blocks Programming mode changes: Fix to correct issue when an exception was thrown because an OpticalDistanceSensor object appears twice in the hardware map (the second time as a LightSensor). Version 2.6 (released on 16.12.16) Fixes for Gyro class: Improve (decrease) sensor refresh latency. fix isCalibrating issues. Blocks Programming mode changes: Blocks now ignores a device in the configuration xml if the name is empty. Other devices work in configuration work fine. Version 2.5 (internal release on released on 16.12.13) Blocks Programming mode changes: Added blocks support for AdafruitBNO055IMU. Added Download Op Mode button to FtcBocks.html. Added support for copying blocks in one OpMode and pasting them in an other OpMode. The clipboard content is stored on the phone, so the programming mode server must be running. Modified Utilities section of the toolbox. In Programming Mode, display information about the active connections. Fixed paste location when workspace has been scrolled. Added blocks support for the android Accelerometer. Fixed issue where Blocks Upload Op Mode truncated name at first dot. Added blocks support for Android SoundPool. Added type safety to blocks for Acceleration. Added type safety to blocks for AdafruitBNO055IMU.Parameters. Added type safety to blocks for AnalogInput. Added type safety to blocks for AngularVelocity. Added type safety to blocks for Color. Added type safety to blocks for ColorSensor. Added type safety to blocks for CompassSensor. Added type safety to blocks for CRServo. Added type safety to blocks for DigitalChannel. Added type safety to blocks for ElapsedTime. Added type safety to blocks for Gamepad. Added type safety to blocks for GyroSensor. Added type safety to blocks for IrSeekerSensor. Added type safety to blocks for LED. Added type safety to blocks for LightSensor. Added type safety to blocks for LinearOpMode. Added type safety to blocks for MagneticFlux. Added type safety to blocks for MatrixF. Added type safety to blocks for MrI2cCompassSensor. Added type safety to blocks for MrI2cRangeSensor. Added type safety to blocks for OpticalDistanceSensor. Added type safety to blocks for Orientation. Added type safety to blocks for Position. Added type safety to blocks for Quaternion. Added type safety to blocks for Servo. Added type safety to blocks for ServoController. Added type safety to blocks for Telemetry. Added type safety to blocks for Temperature. Added type safety to blocks for TouchSensor. Added type safety to blocks for UltrasonicSensor. Added type safety to blocks for VectorF. Added type safety to blocks for Velocity. Added type safety to blocks for VoltageSensor. Added type safety to blocks for VuforiaLocalizer.Parameters. Added type safety to blocks for VuforiaTrackable. Added type safety to blocks for VuforiaTrackables. Added type safety to blocks for enums in AdafruitBNO055IMU.Parameters. Added type safety to blocks for AndroidAccelerometer, AndroidGyroscope, AndroidOrientation, and AndroidTextToSpeech. Version 2.4 (released on 16.11.13) Fix to avoid crashing for nonexistent resources. Blocks Programming mode changes: Added blocks to support OpenGLMatrix, MatrixF, and VectorF. Added blocks to support AngleUnit, AxesOrder, AxesReference, CameraDirection, CameraMonitorFeedback, DistanceUnit, and TempUnit. Added blocks to support Acceleration. Added blocks to support LinearOpMode.getRuntime. Added blocks to support MagneticFlux and Position. Fixed typos. Made blocks for ElapsedTime more consistent with other objects. Added blocks to support Quaternion, Velocity, Orientation, AngularVelocity. Added blocks to support VuforiaTrackables, VuforiaTrackable, VuforiaLocalizer, VuforiaTrackableDefaultListener. Fixed a few blocks. Added type checking to new blocks. Updated to latest blockly. Added default variable blocks to navigation and matrix blocks. Fixed toolbox entry for openGLMatrix_rotation_withAxesArgs. When user downloads Blocks-generated op mode, only the .blk file is downloaded. When user uploads Blocks-generated op mode (.blk file), Javascript code is auto generated. Added DbgLog support. Added logging when a blocks file is read/written. Fixed bug to properly render blocks even if missing devices from configuration file. Added support for additional characters (not just alphanumeric) for the block file names (for download and upload). Added support for OpMode flavor (“Autonomous” or “TeleOp”) and group. Changes to Samples to prevent tutorial issues. Incorporated suggested changes from public pull 216 (“Replace .. paths”). Remove Servo Glitches when robot stopped. if user hits “Cancels” when editing a configuration file, clears the unsaved changes and reverts to original unmodified configuration. Added log info to help diagnose why the Robot Controller app was terminated (for example, by watch dog function). Added ability to transfer log from the controller. Fixed inconsistency for AngularVelocity Limit unbounded growth of data for telemetry. If user does not call telemetry.update() for LinearOpMode in a timely manner, data added for telemetry might get lost if size limit is exceeded. Version 2.35 (released on 16.10.06) Blockly programming mode - Removed unnecesary idle() call from blocks for new project. Version 2.30 (released on 16.10.05) Blockly programming mode: Mechanism added to save Blockly op modes from Programming Mode Server onto local device To avoid clutter, blocks are displayed in categorized folders Added support for DigitalChannel Added support for ModernRoboticsI2cCompassSensor Added support for ModernRoboticsI2cRangeSensor Added support for VoltageSensor Added support for AnalogInput Added support for AnalogOutput Fix for CompassSensor setMode block Vuforia Fix deadlock / make camera data available while Vuforia is running. Update to Vuforia 6.0.117 (recommended by Vuforia and Google to close security loophole). Fix for autonomous 30 second timer bug (where timer was in effect, even though it appeared to have timed out). opModeIsActive changes to allow cleanup after op mode is stopped (with enforced 2 second safety timeout). Fix to avoid reading i2c twice. Updated sample Op Modes. Improved logging and fixed intermittent freezing. Added digital I/O sample. Cleaned up device names in sample op modes to be consistent with Pushbot guide. Fix to allow use of IrSeekerSensorV3. Version 2.20 (released on 16.09.08) Support for Modern Robotics Compass Sensor. Support for Modern Robotics Range Sensor. Revise device names for Pushbot templates to match the names used in Pushbot guide. Fixed bug so that IrSeekerSensorV3 device is accessible as IrSeekerSensor in hardwareMap. Modified computer vision code to require an individual Vuforia license (per legal requirement from PTC). Minor fixes. Blockly enhancements: Support for Voltage Sensor. Support for Analog Input. Support for Analog Output. Support for Light Sensor. Support for Servo Controller. Version 2.10 (released on 16.09.03) Support for Adafruit IMU. Improvements to ModernRoboticsI2cGyro class Block on reset of z axis. isCalibrating() returns true while gyro is calibration. Updated sample gyro program. Blockly enhancements support for android.graphics.Color. added support for ElapsedTime. improved look and legibility of blocks. support for compass sensor. support for ultrasonic sensor. support for IrSeeker. support for LED. support for color sensor. support for CRServo prompt user to configure robot before using programming mode. Provides ability to disable audio cues. various bug fixes and improvements. Version 2.00 (released on 16.08.19) This is the new release for the upcoming 2016-2017 FIRST Tech Challenge Season. Channel change is enabled in the FTC Robot Controller app for Moto G 2nd and 3rd Gen phones. Users can now use annotations to register/disable their Op Modes. Changes in the Android SDK, JDK and build tool requirements (minsdk=19, java 1.7, build tools 23.0.3). Standardized units in analog input. Cleaned up code for existing analog sensor classes. setChannelMode and getChannelMode were REMOVED from the DcMotorController class. This is important - we no longer set the motor modes through the motor controller. setMode and getMode were added to the DcMotor class. ContinuousRotationServo class has been added to the FTC SDK. Range.clip() method has been overloaded so it can support this operation for int, short and byte integers. Some changes have been made (new methods added) on how a user can access items from the hardware map. Users can now set the zero power behavior for a DC motor so that the motor will brake or float when power is zero. Prototype Blockly Programming Mode has been added to FTC Robot Controller. Users can place the Robot Controller into this mode, and then use a device (such as a laptop) that has a Javascript enabled browser to write Blockly-based Op Modes directly onto the Robot Controller. Users can now configure the robot remotely through the FTC Driver Station app. Android Studio project supports Android Studio 2.1.x and compile SDK Version 23 (Marshmallow). Vuforia Computer Vision SDK integrated into FTC SDK. Users can use sample vision targets to get localization information on a standard FTC field. Project structure has been reorganized so that there is now a TeamCode package that users can use to place their local/custom Op Modes into this package. Inspection function has been integrated into the FTC Robot Controller and Driver Station Apps (Thanks Team HazMat… 9277 & 10650!). Audio cues have been incorporated into FTC SDK. Swap mechanism added to FTC Robot Controller configuration activity. For example, if you have two motor controllers on a robot, and you misidentified them in your configuration file, you can use the Swap button to swap the devices within the configuration file (so you do not have to manually re-enter in the configuration info for the two devices). Fix mechanism added to all user to replace an electronic module easily. For example, suppose a servo controller dies on your robot. You replace the broken module with a new module, which has a different serial number from the original servo controller. You can use the Fix button to automatically reconfigure your configuration file to use the serial number of the new module. Improvements made to fix resiliency and responsiveness of the system. For LinearOpMode the user now must for a telemetry.update() to update the telemetry data on the driver station. This update() mechanism ensures that the driver station gets the updated data properly and at the same time. The Auto Configure function of the Robot Controller is now template based. If there is a commonly used robot configuration, a template can be created so that the Auto Configure mechanism can be used to quickly configure a robot of this type. The logic to detect a runaway op mode (both in the LinearOpMode and OpMode types) and to abort the run, then auto recover has been improved/implemented. Fix has been incorporated so that Logitech F310 gamepad mappings will be correct for Marshmallow users. Release 16.07.08 For the ftc_app project, the gradle files have been modified to support Android Studio 2.1.x. Release 16.03.30 For the MIT App Inventor, the design blocks have new icons that better represent the function of each design component. Some changes were made to the shutdown logic to ensure the robust shutdown of some of our USB services. A change was made to LinearOpMode so as to allow a given instance to be executed more than once, which is required for the App Inventor. Javadoc improved/updated. Release 16.03.09 Changes made to make the FTC SDK synchronous (significant change!) waitOneFullHardwareCycle() and waitForNextHardwareCycle() are no longer needed and have been deprecated. runOpMode() (for a LinearOpMode) is now decoupled from the system's hardware read/write thread. loop() (for an OpMode) is now decoupled from the system's hardware read/write thread. Methods are synchronous. For example, if you call setMode(DcMotorController.RunMode.RESET_ENCODERS) for a motor, the encoder is guaranteed to be reset when the method call is complete. For legacy module (NXT compatible), user no longer has to toggle between read and write modes when reading from or writing to a legacy device. Changes made to enhance reliability/robustness during ESD event. Changes made to make code thread safe. Debug keystore added so that user-generated robot controller APKs will all use the same signed key (to avoid conflicts if a team has multiple developer laptops for example). Firmware version information for Modern Robotics modules are now logged. Changes made to improve USB comm reliability and robustness. Added support for voltage indicator for legacy (NXT-compatible) motor controllers. Changes made to provide auto stop capabilities for op modes. A LinearOpMode class will stop when the statements in runOpMode() are complete. User does not have to push the stop button on the driver station. If an op mode is stopped by the driver station, but there is a run away/uninterruptible thread persisting, the app will log an error message then force itself to crash to stop the runaway thread. Driver Station UI modified to display lowest measured voltage below current voltage (12V battery). Driver Station UI modified to have color background for current voltage (green=good, yellow=caution, red=danger, extremely low voltage). javadoc improved (edits and additional classes). Added app build time to About activity for driver station and robot controller apps. Display local IP addresses on Driver Station About activity. Added I2cDeviceSynchImpl. Added I2cDeviceSync interface. Added seconds() and milliseconds() to ElapsedTime for clarity. Added getCallbackCount() to I2cDevice. Added missing clearI2cPortActionFlag. Added code to create log messages while waiting for LinearOpMode shutdown. Fix so Wifi Direct Config activity will no longer launch multiple times. Added the ability to specify an alternate i2c address in software for the Modern Robotics gyro. Release 16.02.09 Improved battery checker feature so that voltage values get refreshed regularly (every 250 msec) on Driver Station (DS) user interface. Improved software so that Robot Controller (RC) is much more resilient and “self-healing” to USB disconnects: If user attempts to start/restart RC with one or more module missing, it will display a warning but still start up. When running an op mode, if one or more modules gets disconnected, the RC & DS will display warnings,and robot will keep on working in spite of the missing module(s). If a disconnected module gets physically reconnected the RC will auto detect the module and the user will regain control of the recently connected module. Warning messages are more helpful (identifies the type of module that’s missing plus its USB serial number). Code changes to fix the null gamepad reference when users try to reference the gamepads in the init() portion of their op mode. NXT light sensor output is now properly scaled. Note that teams might have to readjust their light threshold values in their op modes. On DS user interface, gamepad icon for a driver will disappear if the matching gamepad is disconnected or if that gamepad gets designated as a different driver. Robot Protocol (ROBOCOL) version number info is displayed in About screen on RC and DS apps. Incorporated a display filter on pairing screen to filter out devices that don’t use the “-“ format. This filter can be turned off to show all WiFi Direct devices. Updated text in License file. Fixed formatting error in OpticalDistanceSensor.toString(). Fixed issue on with a blank (“”) device name that would disrupt WiFi Direct Pairing. Made a change so that the WiFi info and battery info can be displayed more quickly on the DS upon connecting to RC. Improved javadoc generation. Modified code to make it easier to support language localization in the future. Release 16.01.04 Updated compileSdkVersion for apps Prevent Wifi from entering power saving mode removed unused import from driver station Corrrected "Dead zone" joystick code. LED.getDeviceName and .getConnectionInfo() return null apps check for ROBOCOL_VERSION mismatch Fix for Telemetry also has off-by-one errors in its data string sizing / short size limitations error User telemetry output is sorted. added formatting variants to DbgLog and RobotLog APIs code modified to allow for a long list of op mode names. changes to improve thread safety of RobocolDatagramSocket Fix for "missing hardware leaves robot controller disconnected from driver station" error fix for "fast tapping of Init/Start causes problems" (toast is now only instantiated on UI thread). added some log statements for thread life cycle. moved gamepad reset logic inside of initActiveOpMode() for robustness changes made to mitigate risk of race conditions on public methods. changes to try and flag when WiFi Direct name contains non-printable characters. fix to correct race condition between .run() and .close() in ReadWriteRunnableStandard. updated FTDI driver made ReadWriteRunnableStanard interface public. fixed off-by-one errors in Command constructor moved specific hardware implmentations into their own package. moved specific gamepad implemnatations to the hardware library. changed LICENSE file to new BSD version. fixed race condition when shutting down Modern Robotics USB devices. methods in the ColorSensor classes have been synchronized. corrected isBusy() status to reflect end of motion. corrected "back" button keycode. the notSupported() method of the GyroSensor class was changed to protected (it should not be public). Release 15.11.04.001 Added Support for Modern Robotics Gyro. The GyroSensor class now supports the MR Gyro Sensor. Users can access heading data (about Z axis) Users can also access raw gyro data (X, Y, & Z axes). Example MRGyroTest.java op mode included. Improved error messages More descriptive error messages for exceptions in user code. Updated DcMotor API Enable read mode on new address in setI2cAddress Fix so that driver station app resets the gamepads when switching op modes. USB-related code changes to make USB comm more responsive and to display more explicit error messages. Fix so that USB will recover properly if the USB bus returns garbage data. Fix USB initializtion race condition. Better error reporting during FTDI open. More explicit messages during USB failures. Fixed bug so that USB device is closed if event loop teardown method was not called. Fixed timer UI issue Fixed duplicate name UI bug (Legacy Module configuration). Fixed race condition in EventLoopManager. Fix to keep references stable when updating gamepad. For legacy Matrix motor/servo controllers removed necessity of appending "Motor" and "Servo" to controller names. Updated HT color sensor driver to use constants from ModernRoboticsUsbLegacyModule class. Updated MR color sensor driver to use constants from ModernRoboticsUsbDeviceInterfaceModule class. Correctly handle I2C Address change in all color sensors Updated/cleaned up op modes. Updated comments in LinearI2cAddressChange.java example op mode. Replaced the calls to "setChannelMode" with "setMode" (to match the new of the DcMotor method). Removed K9AutoTime.java op mode. Added MRGyroTest.java op mode (demonstrates how to use MR Gyro Sensor). Added MRRGBExample.java op mode (demonstrates how to use MR Color Sensor). Added HTRGBExample.java op mode (demonstrates how to use HT legacy color sensor). Added MatrixControllerDemo.java (demonstrates how to use legacy Matrix controller). Updated javadoc documentation. Updated release .apk files for Robot Controller and Driver Station apps. Release 15.10.06.002 Added support for Legacy Matrix 9.6V motor/servo controller. Cleaned up build.gradle file. Minor UI and bug fixes for driver station and robot controller apps. Throws error if Ultrasonic sensor (NXT) is not configured for legacy module port 4 or 5. Release 15.08.03.001 New user interfaces for FTC Driver Station and FTC Robot Controller apps. An init() method is added to the OpMode class. For this release, init() is triggered right before the start() method. Eventually, the init() method will be triggered when the user presses an "INIT" button on driver station. The init() and loop() methods are now required (i.e., need to be overridden in the user's op mode). The start() and stop() methods are optional. A new LinearOpMode class is introduced. Teams can use the LinearOpMode mode to create a linear (not event driven) program model. Teams can use blocking statements like Thread.sleep() within a linear op mode. The API for the Legacy Module and Core Device Interface Module have been updated. Support for encoders with the Legacy Module is now working. The hardware loop has been updated for better performance.
YukunJ / TurtleA C++17-based lightweight high-performance network library
TerbiumOS / WebOSTerbiumOS web is a web os packed with features, with new updates every so often
jettbrains / L W3C Strategic Highlights September 2019 This report was prepared for the September 2019 W3C Advisory Committee Meeting (W3C Member link). See the accompanying W3C Fact Sheet — September 2019. For the previous edition, see the April 2019 W3C Strategic Highlights. For future editions of this report, please consult the latest version. A Chinese translation is available. ☰ Contents Introduction Future Web Standards Meeting Industry Needs Web Payments Digital Publishing Media and Entertainment Web & Telecommunications Real-Time Communications (WebRTC) Web & Networks Automotive Web of Things Strengthening the Core of the Web HTML CSS Fonts SVG Audio Performance Web Performance WebAssembly Testing Browser Testing and Tools WebPlatform Tests Web of Data Web for All Security, Privacy, Identity Internationalization (i18n) Web Accessibility Outreach to the world W3C Developer Relations W3C Training Translations W3C Liaisons Introduction This report highlights recent work of enhancement of the existing landscape of the Web platform and innovation for the growth and strength of the Web. 33 working groups and a dozen interest groups enable W3C to pursue its mission through the creation of Web standards, guidelines, and supporting materials. We track the tremendous work done across the Consortium through homogeneous work-spaces in Github which enables better monitoring and management. We are in the middle of a period where we are chartering numerous working groups which demonstrate the rapid degree of change for the Web platform: After 4 years, we are nearly ready to publish a Payment Request API Proposed Recommendation and we need to soon charter follow-on work. In the last year we chartered the Web Payment Security Interest Group. In the last year we chartered the Web Media Working Group with 7 specifications for next generation Media support on the Web. We have Accessibility Guidelines under W3C Member review which includes Silver, a new approach. We have just launched the Decentralized Identifier Working Group which has tremendous potential because Decentralized Identifier (DID) is an identifier that is globally unique, resolveable with high availability, and cryptographically verifiable. We have Privacy IG (PING) under W3C Member review which strengthens our focus on the tradeoff between privacy and function. We have a new CSS charter under W3C Member review which maps the group's work for the next three years. In this period, W3C and the WHATWG have succesfully completed the negotiation of a Memorandum of Understanding rooted in the mutual belief that that having two distinct specifications claiming to be normative is generally harmful for the Web community. The MOU, signed last May, describes how the two organizations are to collaborate on the development of a single authoritative version of the HTML and DOM specifications. W3C subsequently rechartered the HTML Working Group to assist the W3C community in raising issues and proposing solutions for the HTML and DOM specifications, and for the production of W3C Recommendations from WHATWG Review Drafts. As the Web evolves continuously, some groups are looking for ways for specifications to do so as well. So-called "evergreen recommendations" or "living standards" aim to track continuous development (and maintenance) of features, on a feature-by-feature basis, while getting review and patent commitments. We see the maturation and further development of an incredible number of new technologies coming to the Web. Continued progress in many areas demonstrates the vitality of the W3C and the Web community, as the rest of the report illustrates. Future Web Standards W3C has a variety of mechanisms for listening to what the community thinks could become good future Web standards. These include discussions with the Membership, discussions with other standards bodies, the activities of thousands of participants in over 300 community groups, and W3C Workshops. There are lots of good ideas. The W3C strategy team has been identifying promising topics and invites public participation. Future, recent and under consideration Workshops include: Inclusive XR (5-6 November 2019, Seattle, WA, USA) to explore existing and future approaches on making Virtual and Augmented Reality experiences more inclusive, including to people with disabilities; W3C Workshop on Data Models for Transportation (12-13 September 2019, Palo Alto, CA, USA) W3C Workshop on Web Games (27-28 June 2019, Redmond, WA, USA), view report Second W3C Workshop on the Web of Things (3-5 June 2019, Munich, Germany) W3C Workshop on Web Standardization for Graph Data; Creating Bridges: RDF, Property Graph and SQL (4-6 March 2019, Berlin, Germany), view report Web & Machine Learning. The Strategy Funnel documents the staff's exploration of potential new work at various phases: Exploration and Investigation, Incubation and Evaluation, and eventually to the chartering of a new standards group. The Funnel view is a GitHub Project where new area are issues represented by “cards” which move through the columns, usually from left to right. Most cards start in Exploration and move towards Chartering, or move out of the funnel. Public input is welcome at any stage but particularly once Incubation has begun. This helps W3C identify work that is sufficiently incubated to warrant standardization, to review the ecosystem around the work and indicate interest in participating in its standardization, and then to draft a charter that reflects an appropriate scope. Ongoing feedback can speed up the overall standardization process. Since the previous highlights document, W3C has chartered a number of groups, and started discussion on many more: Newly Chartered or Rechartered Web Application Security WG (03-Apr) Web Payment Security IG (17-Apr) Patent and Standards IG (24-Apr) Web Applications WG (14-May) Web & Networks IG (16-May) Media WG (23-May) Media and Entertainment IG (06-Jun) HTML WG (06-Jun) Decentralized Identifier WG (05-Sep) Extended Privacy IG (PING) (30-Sep) Verifiable Claims WG (30-Sep) Service Workers WG (31-Dec) Dataset Exchange WG (31-Dec) Web of Things Working Group (31-Dec) Web Audio Working Group (31-Dec) Proposed charters / Advance Notice Accessibility Guidelines WG Privacy IG (PING) RDF Literal Direction WG Timed Text WG CSS WG Web Authentication WG Closed Internationalization Tag Set IG Meeting Industry Needs Web Payments All Web Payments specifications W3C's payments standards enable a streamlined checkout experience, enabling a consistent user experience across the Web with lower front end development costs for merchants. Users can store and reuse information and more quickly and accurately complete online transactions. The Web Payments Working Group has republished Payment Request API as a Candidate Recommendation, aiming to publish a Proposed Recommendation in the Fall 2019, and is discussing use cases and features for Payment Request after publication of the 1.0 Recommendation. Browser vendors have been finalizing implementation of features added in the past year (view the implementation report). As work continues on the Payment Handler API and its implementation (currently in Chrome and Edge Canary), one focus in 2019 is to increase adoption in other browsers. Recently, Mastercard demonstrated the use of Payment Request API to carry out EMVCo's Secure Remote Commerce (SRC) protocol whose payment method definition is being developed with active participation by Visa, Mastercard, American Express, and Discover. Payment method availability is a key factor in merchant considerations about adopting Payment Request API. The ability to get uniform adoption of a new payment method such as Secure Remote Commerce (SRC) also depends on the availability of the Payment Handler API in browsers, or of proprietary alternatives. Web Monetization, which the Web Payments Working Group will discuss again at its face-to-face meeting in September, can be used to enable micropayments as an alternative revenue stream to advertising. Since the beginning of 2019, Amazon, Brave Software, JCB, Certus Cybersecurity Solutions and Netflix have joined the Web Payments Working Group. In April, W3C launched the Web Payment Security Group to enable W3C, EMVCo, and the FIDO Alliance to collaborate on a vision for Web payment security and interoperability. Participants will define areas of collaboration and identify gaps between existing technical specifications in order to increase compatibility among different technologies, such as: How do SRC, FIDO, and Payment Request relate? The Payment Services Directive 2 (PSD2) regulations in Europe are scheduled to take effect in September 2019. What is the role of EMVCo, W3C, and FIDO technologies, and what is the current state of readiness for the deadline? How can we improve privacy on the Web at the same time as we meet industry requirements regarding user identity? Digital Publishing All Digital Publishing specifications, Publication milestones The Web is the universal publishing platform. Publishing is increasingly impacted by the Web, and the Web increasingly impacts Publishing. Topic of particular interest to Publishing@W3C include typography and layout, accessibility, usability, portability, distribution, archiving, offline access, print on demand, and reliable cross referencing. And the diverse publishing community represented in the groups consist of the traditional "trade" publishers, ebook reading system manufacturers, but also publishers of audio book, scholarly journals or educational materials, library scientists or browser developers. The Publishing Working Group currently concentrates on Audiobooks which lack a comprehensive standard, thus incurring extra costs and time to publish in this booming market. Active development is ongoing on the future standard: Publication Manifest Audiobook profile for Web Publications Lightweight Packaging Format The BD Comics Manga Community Group, the Synchronized Multimedia for Publications Community Group, the Publishing Community Group and a future group on archival, are companions to the working group where specific work is developed and incubated. The Publishing Community Group is a recently launched incubation channel for Publishing@W3C. The goal of the group is to propose, document, and prototype features broadly related to: publications on the Web reading modes and systems and the user experience of publications The EPUB 3 Community Group has successfully completed the revision of EPUB 3.2. The Publishing Business Group fosters ongoing participation by members of the publishing industry and the overall ecosystem in the development of Web infrastructure to better support the needs of the industry. The Business Group serves as an additional conduit to the Publishing Working Group and several Community Groups for feedback between the publishing ecosystem and W3C. The Publishing BG has played a vital role in fostering and advancing the adoption and continued development of EPUB 3. In particular the BG provided critical support to the update of EPUBCheck to validate EPUB content to the new EPUB 3.2 specification. This resulted in the development, in conjunction with the EPUB3 Community Group, of a new generation of EPUBCheck, i.e., EPUBCheck 4.2 production-ready release. Media and Entertainment All Media specifications The Media and Entertainment vertical tracks media-related topics and features that create immersive experiences for end users. HTML5 brought standard audio and video elements to the Web. Standardization activities since then have aimed at turning the Web into a professional platform fully suitable for the delivery of media content and associated materials, enabling missing features to stream video content on the Web such as adaptive streaming and content protection. Together with Microsoft, Comcast, Netflix and Google, W3C received an Technology & Engineering Emmy Award in April 2019 for standardization of a full TV experience on the Web. Current goals are to: Reinforce core media technologies: Creation of the Media Working Group, to develop media-related specifications incubated in the WICG (e.g. Media Capabilities, Picture-in-picture, Media Session) and maintain maintain/evolve Media Source Extensions (MSE) and Encrypted Media Extensions (EME). Improve support for Media Timed Events: data cues incubation. Enhance color support (HDR, wide gamut), in scope of the CSS WG and in the Color on the Web CG. Reduce fragmentation: Continue annual releases of a common and testable baseline media devices, in scope of the Web Media APIs CG and in collaboration with the CTA WAVE Project. Maintain the Road-map of Media Technologies for the Web which highlights Web technologies that can be used to build media applications and services, as well as known gaps to enable additional use cases. Create the future: Discuss perspectives for Media and Entertainment for the Web. Bring the power of GPUs to the Web (graphics, machine learning, heavy processing), under incubation in the GPU for the Web CG. Transition to a Working Group is under discussion. Determine next steps after the successful W3C Workshop on Web Games of June 2019. View the report. Timed Text The Timed Text Working Group develops and maintains formats used for the representation of text synchronized with other timed media, like audio and video, and notably works on TTML, profiles of TTML, and WebVTT. Recent progress includes: A robust WebVTT implementation report poises the specification for publication as a proposed recommendation. Discussions around re-chartering, notably to add a TTML Profile for Audio Description deliverable to the scope of the group, and clarify that rendering of captions within XR content is also in scope. Immersive Web Hardware that enables Virtual Reality (VR) and Augmented Reality (AR) applications are now broadly available to consumers, offering an immersive computing platform with both new opportunities and challenges. The ability to interact directly with immersive hardware is critical to ensuring that the web is well equipped to operate as a first-class citizen in this environment. The Immersive Web Working Group has been stabilizing the WebXR Device API while the companion Immersive Web Community Group incubates the next series of features identified as key for the future of the Immersive Web. W3C plans a workshop focused on the needs and benefits at the intersection of VR & Accessibility (Inclusive XR), on 5-6 November 2019 in Seattle, WA, USA, to explore existing and future approaches on making Virtual and Augmented Reality experiences more inclusive. Web & Telecommunications The Web is the Open Platform for Mobile. Telecommunication service providers and network equipment providers have long been critical actors in the deployment of Web technologies. As the Web platform matures, it brings richer and richer capabilities to extend existing services to new users and devices, and propose new and innovative services. Real-Time Communications (WebRTC) All Real-Time Communications specifications WebRTC has reshaped the whole communication landscape by making any connected device a potential communication end-point, bringing audio and video communications anywhere, on any network, vastly expanding the ability of operators to reach their customers. WebRTC serves as the corner-stone of many online communication and collaboration services. The WebRTC Working Group aims to bringing WebRTC 1.0 (and companion specification Media Capture and Streams) to Recommendation by the end of 2019. Intense efforts are focused on testing (supported by a dedicated hackathon at IETF 104) and interoperability. The group is considering pushing features that have not gotten enough traction to separate modules or to a later minor revision of the spec. Beyond WebRTC 1.0, the WebRTC Working Group will focus its efforts on WebRTC NV which the group has started documenting by identifying use cases. Web & Networks Recently launched, in the wake of the May 2018 Web5G workshop, the Web & Networks Interest Group is chaired by representatives from AT&T, China Mobile and Intel, with a goal to explore solutions for web applications to achieve better performance and resource allocation, both on the device and network. The group's first efforts are around use cases, privacy & security requirements and liaisons. Automotive All Automotive specifications To create a rich application ecosystem for vehicles and other devices allowed to connect to the vehicle, the W3C Automotive Working Group is delivering a service specification to expose all common vehicle signals (engine temperature, fuel/charge level, range, tire pressure, speed, etc.) The Vehicle Information Service Specification (VISS), which is a Candidate Recommendation, is seeing more implementations across the industry. It provides the access method to a common data model for all the vehicle signals –presently encapsulating a thousand or so different data elements– and will be growing to accommodate the advances in automotive such as autonomous and driver assist technologies and electrification. The group is already working on a successor to VISS, leveraging the underlying data model and the VIWI submission from Volkswagen, for a more robust means of accessing vehicle signals information and the same paradigm for other automotive needs including location-based services, media, notifications and caching content. The Automotive and Web Platform Business Group acts as an incubator for prospective standards work. One of its task forces is using W3C VISS in performing data sampling and off-boarding the information to the cloud. Access to the wealth of information that W3C's auto signals standard exposes is of interest to regulators, urban planners, insurance companies, auto manufacturers, fleet managers and owners, service providers and others. In addition to components needed for data sampling and edge computing, capturing user and owner consent, information collection methods and handling of data are in scope. The upcoming W3C Workshop on Data Models for Transportation (September 2019) is expected to focus on the need of additional ontologies around transportation space. Web of Things All Web of Things specifications W3C's Web of Things work is designed to bridge disparate technology stacks to allow devices to work together and achieve scale, thus enabling the potential of the Internet of Things by eliminating fragmentation and fostering interoperability. Thing descriptions expressed in JSON-LD cover the behavior, interaction affordances, data schema, security configuration, and protocol bindings. The Web of Things complements existing IoT ecosystems to reduce the cost and risk for suppliers and consumers of applications that create value by combining multiple devices and information services. There are many sectors that will benefit, e.g. smart homes, smart cities, smart industry, smart agriculture, smart healthcare and many more. The Web of Things Working Group is finishing the initial Web of Things standards, with support from the Web of Things Interest Group: Web of Things Architecture Thing Descriptions Strengthening the Core of the Web HTML The HTML Working Group was chartered early June to assist the W3C community in raising issues and proposing solutions for the HTML and DOM specifications, and to produce W3C Recommendations from WHATWG Review Drafts. A few days before, W3C and the WHATWG signed a Memorandum of Understanding outlining the agreement to collaborate on the development of a single version of the HTML and DOM specifications. Issues and proposed solutions for HTML and DOM done via the newly rechartered HTML Working Group in the WHATWG repositories The HTML Working Group is targetting November 2019 to bring HTML and DOM to Candidate Recommendations. CSS All CSS specifications CSS is a critical part of the Open Web Platform. The CSS Working Group gathers requirements from two large groups of CSS users: the publishing industry and application developers. Within W3C, those groups are exemplified by the Publishing groups and the Web Platform Working Group. The former requires things like better pagination support and advanced font handling, the latter needs intelligent (and fast!) scrolling and animations. What we know as CSS is actually a collection of almost a hundred specifications, referred to as ‘modules’. The current state of CSS is defined by a snapshot, updated once a year. The group also publishes an index defining every term defined by CSS specifications. Fonts All Fonts specifications The Web Fonts Working Group develops specifications that allow the interoperable deployment of downloadable fonts on the Web, with a focus on Progressive Font Enrichment as well as maintenance of WOFF Recommendations. Recent and ongoing work includes: Early API experiments by Adobe and Monotype have demonstrated the feasibility of a font enrichment API, where a server delivers a font with minimal glyph repertoire and the client can query the full repertoire and request additional subsets on-the-fly. In other experiments, the Brotli compression used in WOFF 2 was extended to support shared dictionaries and patch update. Metrics to quantify improvement are a current hot discussion topic. The group will meet at ATypi 2019 in Japan, to gather requirements from the international typography community. The group will first produce a report summarizing the strengths and weaknesses of each prototype solution by Q2 2020. SVG All SVG specifications SVG is an important and widely-used part of the Open Web Platform. The SVG Working Group focuses on aligning the SVG 2.0 specification with browser implementations, having split the specification into a currently-implemented 2.0 and a forward-looking 2.1. Current activity is on stabilization, increased integration with the Open Web Platform, and test coverage analysis. The Working Group was rechartered in March 2019. A new work item concerns native (non-Web-browser) uses of SVG as a non-interactive, vector graphics format. Audio The Web Audio Working Group was extended to finish its work on the Web Audio API, expecting to publish it as a Recommendation by year end. The specification enables synthesizing audio in the browser. Audio operations are performed with audio nodes, which are linked together to form a modular audio routing graph. Multiple sources — with different types of channel layout — are supported. This modular design provides the flexibility to create complex audio functions with dynamic effects. The first version of Web Audio API is now feature complete and is implemented in all modern browsers. Work has started on the next version, and new features are being incubated in the Audio Community Group. Performance Web Performance All Web Performance specifications There are currently 18 specifications in development in the Web Performance Working Group aiming to provide methods to observe and improve aspects of application performance of user agent features and APIs. The W3C team is looking at related work incubated in the W3C GPU for the Web (WebGPU) Community Group which is poised to transition to a W3C Working Group. A preliminary draft charter is available. WebAssembly All WebAssembly specifications WebAssembly improves Web performance and power by being a virtual machine and execution environment enabling loaded pages to run native (compiled) code. It is deployed in Firefox, Edge, Safari and Chrome. The specification will soon reach Candidate Recommendation. WebAssembly enables near-native performance, optimized load time, and perhaps most importantly, a compilation target for existing code bases. While it has a small number of native types, much of the performance increase relative to Javascript derives from its use of consistent typing. WebAssembly leverages decades of optimization for compiled languages and the byte code is optimized for compactness and streaming (the web page starts executing while the rest of the code downloads). Network and API access all occurs through accompanying Javascript libraries -- the security model is identical to that of Javascript. Requirements gathering and language development occur in the Community Group while the Working Group manages test development, community review and progression of specifications on the Recommendation Track. Testing Browser testing plays a critical role in the growth of the Web by: Improving the reliability of Web technology definitions; Improving the quality of implementations of these technologies by helping vendors to detect bugs in their products; Improving the data available to Web developers on known bugs and deficiencies of Web technologies by publishing results of these tests. Browser Testing and Tools The Browser Testing and Tools Working Group is developing WebDriver version 2, having published last year the W3C Recommendation of WebDriver. WebDriver acts as a remote control interface that enables introspection and control of user agents, provides a platform- and language-neutral wire protocol as a way for out-of-process programs to remotely instruct the behavior of Web, and emulates the actions of a real person using the browser. WebPlatform Tests The WebPlatform Tests project now provides a mechanism which allows to fully automate tests that previously needed to be run manually: TestDriver. TestDriver enables sending trusted key and mouse events, sending complex series of trusted pointer and key interactions for things like in-content drag-and-drop or pinch zoom, and even file upload. Since 2014 W3C began work on this coordinated open-source effort to build a cross-browser test suite for the Web Platform, which WHATWG, and all major browsers adopted. Web of Data All Data specifications There have been several great success stories around the standardization of data on the web over the past year. Verifiable Claims seems to have significant uptake. It is also significant that the Distributed Identifier WG charter has received numerous favorable reviews, and was just recently launched. JSON-LD has been a major success with the large deployment on Web sites via schema.org. JSON-LD 1.1 completed technical work, about to transition to CR More than 25% of websites today include schema.org data in JSON-LD The Web of Things description is in CR since May, making use of JSON-LD Verifiable Credentials data model is in CR since July, also making use of JSON-LD Continued strong interest in decentralized identifiers Engagement from the TAG with reframing core documents, such as Ethical Web Principles, to include data on the web within their scope Data is increasingly important for all organizations, especially with the rise of IoT and Big Data. W3C has a mature and extensive suite of standards relating to data that were developed over two decades of experience, with plans for further work on making it easier for developers to work with graph data and knowledge graphs. Linked Data is about the use of URIs as names for things, the ability to dereference these URIs to get further information and to include links to other data. There are ever-increasing sources of open Linked Data on the Web, as well as data services that are restricted to the suppliers and consumers of those services. The digital transformation of industry is seeking to exploit advanced digital technologies. This will facilitate businesses to integrate horizontally along the supply and value chains, and vertically from the factory floor to the office floor. W3C is seeking to make it easier to support enterprise-wide data management and governance, reflecting the strategic importance of data to modern businesses. Traditional approaches to data have focused on tabular databases (SQL/RDBMS), Comma Separated Value (CSV) files, and data embedded in PDF documents and spreadsheets. We're now in midst of a major shift to graph data with nodes and labeled directed links between them. Graph data is: Faster than using SQL and associated JOIN operations More favorable to integrating data from heterogeneous sources Better suited to situations where the data model is evolving In the wake of the recent W3C Workshop on Graph Data we are in the process of launching a Graph Standardization Business Group to provide a business perspective with use cases and requirements, to coordinate technical standards work and liaisons with external organizations. Web for All Security, Privacy, Identity All Security specifications, all Privacy specifications Authentication on the Web As the WebAuthn Level 1 W3C Recommendation published last March is seeing wide implementation and adoption of strong cryptographic authentication, work is proceeding on Level 2. The open standard Web API gives native authentication technology built into native platforms, browsers, operating systems (including mobile) and hardware, offering protection against hacking, credential theft, phishing attacks, thus aiming to end the era of passwords as a security construct. You may read more in our March press release. Privacy An increasing number of W3C specifications are benefitting from Privacy and Security review; there are security and privacy aspects to every specification. Early review is essential. Working with the TAG, the Privacy Interest Group has updated the Self-Review Questionnaire: Security and Privacy. Other recent work of the group includes public blogging further to the exploration of anti-patterns in standards and permission prompts. Security The Web Application Security Working Group adopted Feature Policy, aiming to allow developers to selectively enable, disable, or modify the behavior of some of these browser features and APIs within their application; and Fetch Metadata, aiming to provide servers with enough information to make a priori decisions about whether or not to service a request based on the way it was made, and the context in which it will be used. The Web Payment Security Interest Group, launched last April, convenes members from W3C, EMVCo, and the FIDO Alliance to discuss cooperative work to enhance the security and interoperability of Web payments (read more about payments). Internationalization (i18n) All Internationalization specifications, educational articles related to Internationalization, spec developers checklist Only a quarter or so current Web users use English online and that proportion will continue to decrease as the Web reaches more and more communities of limited English proficiency. If the Web is to live up to the "World Wide" portion of its name, and for the Web to truly work for stakeholders all around the world engaging with content in various languages, it must support the needs of worldwide users as they engage with content in the various languages. The growth of epublishing also brings requirements for new features and improved typography on the Web. It is important to ensure the needs of local communities are captured. The W3C Internationalization Initiative was set up to increase in-house resources dedicated to accelerating progress in making the World Wide Web "worldwide" by gathering user requirements, supporting developers, and education & outreach. For an overview of current projects see the i18n radar. W3C's Internationalization efforts progressed on a number of fronts recently: Requirements: New African and European language groups will work on the gap analysis, errata and layout requirements. Gap analysis: Japanese, Devanagari, Bengali, Tamil, Lao, Khmer, Javanese, and Ethiopic updated in the gap-analysis documents. Layout requirements document: notable progress tracked in the Southeast Asian Task Force while work continues on Chinese layout requirements. Developer support: Spec reviews: the i18n WG continues active review of specifications of the WHATWG and other W3C Working Groups. Short review checklist: easy way to begin a self-review to help spec developers understand what aspects of their spec are likely to need attention for internationalization, and points them to more detailed checklists for the relevant topics. It also helps those reviewing specs for i18n issues. Strings on the Web: Language and Direction Metadata lays out issues and discusses potential solutions for passing information about language and direction with strings in JSON or other data formats. The document was rewritten for clarity, and expanded. The group is collaborating with the JSON-LD and Web Publishing groups to develop a plan for updating RDF, JSON-LD and related specifications to handle metadata for base direction of text (bidi). User-friendly test format: a new format was developed for Internationalization Test Suite tests, which displays helpful information about how the test works. This particularly useful because those tests are pointed to by educational materials and gap-analysis documents. Web Platform Tests: a large number of tests in the i18n test suite have been ported to the WPT repository, including: css-counter-styles, css-ruby, css-syntax, css-test, css-text-decor, css-writing-modes, and css-pseudo. Education & outreach: (for all educational materials, see the HTML & CSS Authoring Techniques) Web Accessibility All Accessibility specifications, WAI resources The Web Accessibility Initiative supports W3C's Web for All mission. Recent achievements include: Education and training: Inaccessibility of CAPTCHA updated to bring our analysis and recommendations up to date with CAPTCHA practice today, concluding two years of extensive work and invaluable input from the public (read more on the W3C Blog Learn why your web content and applications should be accessible. The Education and Outreach Working Group has completed revision and updating of the Business Case for Digital Accessibility. Accessibility guidelines: The Accessibility Guidelines Working Group has continued to update WCAG Techniques and Understanding WCAG 2.1; and published a Candidate Recommendation of Accessibility Conformance Testing Rules Format 1.0 to improve inter-rater reliability when evaluating conformance of web content to WCAG An updated charter is being developed to host work on "Silver", the next generation accessibility guidelines (WCAG 2.2) There are accessibility aspects to most specifications. Check your work with the FAST checklist. Outreach to the world W3C Developer Relations To foster the excellent feedback loop between Web Standards development and Web developers, and to grow participation from that diverse community, recent W3C Developer Relations activities include: @w3cdevs tracks the enormous amount of work happening across W3C W3C Track during the Web Conference 2019 in San Francisco Tech videos: W3C published the 2019 Web Games Workshop videos The 16 September 2019 Developer Meetup in Fukuoka, Japan, is open to all and will combine a set of technical demos prepared by W3C groups, and a series of talks on a selected set of W3C technologies and projects W3C is involved with Mozilla, Google, Samsung, Microsoft and Bocoup in the organization of ViewSource 2019 in Amsterdam (read more on the W3C Blog) W3C Training In partnership with EdX, W3C's MOOC training program, W3Cx offers a complete "Front-End Web Developer" (FEWD) professional certificate program that consists of a suite of five courses on the foundational languages that power the Web: HTML5, CSS and JavaScript. We count nearly 900K students from all over the world. Translations Many Web users rely on translations of documents developed at W3C whose official language is English. W3C is extremely grateful to the continuous efforts of its community in ensuring our various deliverables in general, and in our specifications in particular, are made available in other languages, for free, ensuring their exposure to a much more diverse set of readers. Last Spring we developed a more robust system, a new listing of translations of W3C specifications and updated the instructions on how to contribute to our translation efforts. W3C Liaisons Liaisons and coordination with numerous organizations and Standards Development Organizations (SDOs) is crucial for W3C to: make sure standards are interoperable coordinate our respective agenda in Internet governance: W3C participates in ICANN, GIPO, IGF, the I* organizations (ICANN, IETF, ISOC, IAB). ensure at the government liaison level that our standards work is officially recognized when important to our membership so that products based on them (often done by our members) are part of procurement orders. W3C has ARO/PAS status with ISO. W3C participates in the EU MSP and Rolling Plan on Standardization ensure the global set of Web and Internet standards form a compatible stack of technologies, at the technical and policy level (patent regime, fragmentation, use in policy making) promote Standards adoption equally by the industry, the public sector, and the public at large Coralie Mercier, Editor, W3C Marketing & Communications $Id: Overview.html,v 1.60 2019/10/15 12:05:52 coralie Exp $ Copyright © 2019 W3C ® (MIT, ERCIM, Keio, Beihang) Usage policies apply.
sanusanth / Python Basic ProgramsWhat is Python? Executive Summary Python is an interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together. Python's simple, easy to learn syntax emphasizes readability and therefore reduces the cost of program maintenance. Python supports modules and packages, which encourages program modularity and code reuse. The Python interpreter and the extensive standard library are available in source or binary form without charge for all major platforms, and can be freely distributed. Often, programmers fall in love with Python because of the increased productivity it provides. Since there is no compilation step, the edit-test-debug cycle is incredibly fast. Debugging Python programs is easy: a bug or bad input will never cause a segmentation fault. Instead, when the interpreter discovers an error, it raises an exception. When the program doesn't catch the exception, the interpreter prints a stack trace. A source level debugger allows inspection of local and global variables, evaluation of arbitrary expressions, setting breakpoints, stepping through the code a line at a time, and so on. The debugger is written in Python itself, testifying to Python's introspective power. On the other hand, often the quickest way to debug a program is to add a few print statements to the source: the fast edit-test-debug cycle makes this simple approach very effective. What is Python? Python is a popular programming language. It was created by Guido van Rossum, and released in 1991. It is used for: web development (server-side), software development, mathematics, system scripting. What can Python do? Python can be used on a server to create web applications. Python can be used alongside software to create workflows. Python can connect to database systems. It can also read and modify files. Python can be used to handle big data and perform complex mathematics. Python can be used for rapid prototyping, or for production-ready software development. Why Python? Python works on different platforms (Windows, Mac, Linux, Raspberry Pi, etc). Python has a simple syntax similar to the English language. Python has syntax that allows developers to write programs with fewer lines than some other programming languages. Python runs on an interpreter system, meaning that code can be executed as soon as it is written. This means that prototyping can be very quick. Python can be treated in a procedural way, an object-oriented way or a functional way. Good to know The most recent major version of Python is Python 3, which we shall be using in this tutorial. However, Python 2, although not being updated with anything other than security updates, is still quite popular. In this tutorial Python will be written in a text editor. It is possible to write Python in an Integrated Development Environment, such as Thonny, Pycharm, Netbeans or Eclipse which are particularly useful when managing larger collections of Python files. Python Syntax compared to other programming languages Python was designed for readability, and has some similarities to the English language with influence from mathematics. Python uses new lines to complete a command, as opposed to other programming languages which often use semicolons or parentheses. Python relies on indentation, using whitespace, to define scope; such as the scope of loops, functions and classes. Other programming languages often use curly-brackets for this purpose. Applications for Python Python is used in many application domains. Here's a sampling. The Python Package Index lists thousands of third party modules for Python. Web and Internet Development Python offers many choices for web development: Frameworks such as Django and Pyramid. Micro-frameworks such as Flask and Bottle. Advanced content management systems such as Plone and django CMS. Python's standard library supports many Internet protocols: HTML and XML JSON E-mail processing. Support for FTP, IMAP, and other Internet protocols. Easy-to-use socket interface. And the Package Index has yet more libraries: Requests, a powerful HTTP client library. Beautiful Soup, an HTML parser that can handle all sorts of oddball HTML. Feedparser for parsing RSS/Atom feeds. Paramiko, implementing the SSH2 protocol. Twisted Python, a framework for asynchronous network programming. Scientific and Numeric Python is widely used in scientific and numeric computing: SciPy is a collection of packages for mathematics, science, and engineering. Pandas is a data analysis and modeling library. IPython is a powerful interactive shell that features easy editing and recording of a work session, and supports visualizations and parallel computing. The Software Carpentry Course teaches basic skills for scientific computing, running bootcamps and providing open-access teaching materials. Education Python is a superb language for teaching programming, both at the introductory level and in more advanced courses. Books such as How to Think Like a Computer Scientist, Python Programming: An Introduction to Computer Science, and Practical Programming. The Education Special Interest Group is a good place to discuss teaching issues. Desktop GUIs The Tk GUI library is included with most binary distributions of Python. Some toolkits that are usable on several platforms are available separately: wxWidgets Kivy, for writing multitouch applications. Qt via pyqt or pyside Platform-specific toolkits are also available: GTK+ Microsoft Foundation Classes through the win32 extensions Software Development Python is often used as a support language for software developers, for build control and management, testing, and in many other ways. SCons for build control. Buildbot and Apache Gump for automated continuous compilation and testing. Roundup or Trac for bug tracking and project management. Business Applications Python is also used to build ERP and e-commerce systems: Odoo is an all-in-one management software that offers a range of business applications that form a complete suite of enterprise management applications. Try ton is a three-tier high-level general purpose application platform.
rramatchandran / Big O Performance Java# big-o-performance A simple html app to demonstrate performance costs of data structures. - Clone the project - Navigate to the root of the project in a termina or command prompt - Run 'npm install' - Run 'npm start' - Go to the URL specified in the terminal or command prompt to try out the app. # This app was created from the Create React App NPM. Below are instructions from that project. Below you will find some information on how to perform common tasks. You can find the most recent version of this guide [here](https://github.com/facebookincubator/create-react-app/blob/master/template/README.md). ## Table of Contents - [Updating to New Releases](#updating-to-new-releases) - [Sending Feedback](#sending-feedback) - [Folder Structure](#folder-structure) - [Available Scripts](#available-scripts) - [npm start](#npm-start) - [npm run build](#npm-run-build) - [npm run eject](#npm-run-eject) - [Displaying Lint Output in the Editor](#displaying-lint-output-in-the-editor) - [Installing a Dependency](#installing-a-dependency) - [Importing a Component](#importing-a-component) - [Adding a Stylesheet](#adding-a-stylesheet) - [Post-Processing CSS](#post-processing-css) - [Adding Images and Fonts](#adding-images-and-fonts) - [Adding Bootstrap](#adding-bootstrap) - [Adding Flow](#adding-flow) - [Adding Custom Environment Variables](#adding-custom-environment-variables) - [Integrating with a Node Backend](#integrating-with-a-node-backend) - [Proxying API Requests in Development](#proxying-api-requests-in-development) - [Deployment](#deployment) - [Now](#now) - [Heroku](#heroku) - [Surge](#surge) - [GitHub Pages](#github-pages) - [Something Missing?](#something-missing) ## Updating to New Releases Create React App is divided into two packages: * `create-react-app` is a global command-line utility that you use to create new projects. * `react-scripts` is a development dependency in the generated projects (including this one). You almost never need to update `create-react-app` itself: it’s delegates all the setup to `react-scripts`. When you run `create-react-app`, it always creates the project with the latest version of `react-scripts` so you’ll get all the new features and improvements in newly created apps automatically. To update an existing project to a new version of `react-scripts`, [open the changelog](https://github.com/facebookincubator/create-react-app/blob/master/CHANGELOG.md), find the version you’re currently on (check `package.json` in this folder if you’re not sure), and apply the migration instructions for the newer versions. In most cases bumping the `react-scripts` version in `package.json` and running `npm install` in this folder should be enough, but it’s good to consult the [changelog](https://github.com/facebookincubator/create-react-app/blob/master/CHANGELOG.md) for potential breaking changes. We commit to keeping the breaking changes minimal so you can upgrade `react-scripts` painlessly. ## Sending Feedback We are always open to [your feedback](https://github.com/facebookincubator/create-react-app/issues). ## Folder Structure After creation, your project should look like this: ``` my-app/ README.md index.html favicon.ico node_modules/ package.json src/ App.css App.js index.css index.js logo.svg ``` For the project to build, **these files must exist with exact filenames**: * `index.html` is the page template; * `favicon.ico` is the icon you see in the browser tab; * `src/index.js` is the JavaScript entry point. You can delete or rename the other files. You may create subdirectories inside `src`. For faster rebuilds, only files inside `src` are processed by Webpack. You need to **put any JS and CSS files inside `src`**, or Webpack won’t see them. You can, however, create more top-level directories. They will not be included in the production build so you can use them for things like documentation. ## Available Scripts In the project directory, you can run: ### `npm start` Runs the app in the development mode.<br> Open [http://localhost:3000](http://localhost:3000) to view it in the browser. The page will reload if you make edits.<br> You will also see any lint errors in the console. ### `npm run build` Builds the app for production to the `build` folder.<br> It correctly bundles React in production mode and optimizes the build for the best performance. The build is minified and the filenames include the hashes.<br> Your app is ready to be deployed! ### `npm run eject` **Note: this is a one-way operation. Once you `eject`, you can’t go back!** If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project. Instead, it will copy all the configuration files and the transitive dependencies (Webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own. You don’t have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it. ## Displaying Lint Output in the Editor >Note: this feature is available with `react-scripts@0.2.0` and higher. Some editors, including Sublime Text, Atom, and Visual Studio Code, provide plugins for ESLint. They are not required for linting. You should see the linter output right in your terminal as well as the browser console. However, if you prefer the lint results to appear right in your editor, there are some extra steps you can do. You would need to install an ESLint plugin for your editor first. >**A note for Atom `linter-eslint` users** >If you are using the Atom `linter-eslint` plugin, make sure that **Use global ESLint installation** option is checked: ><img src="http://i.imgur.com/yVNNHJM.png" width="300"> Then make sure `package.json` of your project ends with this block: ```js { // ... "eslintConfig": { "extends": "./node_modules/react-scripts/config/eslint.js" } } ``` Projects generated with `react-scripts@0.2.0` and higher should already have it. If you don’t need ESLint integration with your editor, you can safely delete those three lines from your `package.json`. Finally, you will need to install some packages *globally*: ```sh npm install -g eslint babel-eslint eslint-plugin-react eslint-plugin-import eslint-plugin-jsx-a11y eslint-plugin-flowtype ``` We recognize that this is suboptimal, but it is currently required due to the way we hide the ESLint dependency. The ESLint team is already [working on a solution to this](https://github.com/eslint/eslint/issues/3458) so this may become unnecessary in a couple of months. ## Installing a Dependency The generated project includes React and ReactDOM as dependencies. It also includes a set of scripts used by Create React App as a development dependency. You may install other dependencies (for example, React Router) with `npm`: ``` npm install --save <library-name> ``` ## Importing a Component This project setup supports ES6 modules thanks to Babel. While you can still use `require()` and `module.exports`, we encourage you to use [`import` and `export`](http://exploringjs.com/es6/ch_modules.html) instead. For example: ### `Button.js` ```js import React, { Component } from 'react'; class Button extends Component { render() { // ... } } export default Button; // Don’t forget to use export default! ``` ### `DangerButton.js` ```js import React, { Component } from 'react'; import Button from './Button'; // Import a component from another file class DangerButton extends Component { render() { return <Button color="red" />; } } export default DangerButton; ``` Be aware of the [difference between default and named exports](http://stackoverflow.com/questions/36795819/react-native-es-6-when-should-i-use-curly-braces-for-import/36796281#36796281). It is a common source of mistakes. We suggest that you stick to using default imports and exports when a module only exports a single thing (for example, a component). That’s what you get when you use `export default Button` and `import Button from './Button'`. Named exports are useful for utility modules that export several functions. A module may have at most one default export and as many named exports as you like. Learn more about ES6 modules: * [When to use the curly braces?](http://stackoverflow.com/questions/36795819/react-native-es-6-when-should-i-use-curly-braces-for-import/36796281#36796281) * [Exploring ES6: Modules](http://exploringjs.com/es6/ch_modules.html) * [Understanding ES6: Modules](https://leanpub.com/understandinges6/read#leanpub-auto-encapsulating-code-with-modules) ## Adding a Stylesheet This project setup uses [Webpack](https://webpack.github.io/) for handling all assets. Webpack offers a custom way of “extending” the concept of `import` beyond JavaScript. To express that a JavaScript file depends on a CSS file, you need to **import the CSS from the JavaScript file**: ### `Button.css` ```css .Button { padding: 20px; } ``` ### `Button.js` ```js import React, { Component } from 'react'; import './Button.css'; // Tell Webpack that Button.js uses these styles class Button extends Component { render() { // You can use them as regular CSS styles return <div className="Button" />; } } ``` **This is not required for React** but many people find this feature convenient. You can read about the benefits of this approach [here](https://medium.com/seek-ui-engineering/block-element-modifying-your-javascript-components-d7f99fcab52b). However you should be aware that this makes your code less portable to other build tools and environments than Webpack. In development, expressing dependencies this way allows your styles to be reloaded on the fly as you edit them. In production, all CSS files will be concatenated into a single minified `.css` file in the build output. If you are concerned about using Webpack-specific semantics, you can put all your CSS right into `src/index.css`. It would still be imported from `src/index.js`, but you could always remove that import if you later migrate to a different build tool. ## Post-Processing CSS This project setup minifies your CSS and adds vendor prefixes to it automatically through [Autoprefixer](https://github.com/postcss/autoprefixer) so you don’t need to worry about it. For example, this: ```css .App { display: flex; flex-direction: row; align-items: center; } ``` becomes this: ```css .App { display: -webkit-box; display: -ms-flexbox; display: flex; -webkit-box-orient: horizontal; -webkit-box-direction: normal; -ms-flex-direction: row; flex-direction: row; -webkit-box-align: center; -ms-flex-align: center; align-items: center; } ``` There is currently no support for preprocessors such as Less, or for sharing variables across CSS files. ## Adding Images and Fonts With Webpack, using static assets like images and fonts works similarly to CSS. You can **`import` an image right in a JavaScript module**. This tells Webpack to include that image in the bundle. Unlike CSS imports, importing an image or a font gives you a string value. This value is the final image path you can reference in your code. Here is an example: ```js import React from 'react'; import logo from './logo.png'; // Tell Webpack this JS file uses this image console.log(logo); // /logo.84287d09.png function Header() { // Import result is the URL of your image return <img src={logo} alt="Logo" />; } export default function Header; ``` This works in CSS too: ```css .Logo { background-image: url(./logo.png); } ``` Webpack finds all relative module references in CSS (they start with `./`) and replaces them with the final paths from the compiled bundle. If you make a typo or accidentally delete an important file, you will see a compilation error, just like when you import a non-existent JavaScript module. The final filenames in the compiled bundle are generated by Webpack from content hashes. If the file content changes in the future, Webpack will give it a different name in production so you don’t need to worry about long-term caching of assets. Please be advised that this is also a custom feature of Webpack. **It is not required for React** but many people enjoy it (and React Native uses a similar mechanism for images). However it may not be portable to some other environments, such as Node.js and Browserify. If you prefer to reference static assets in a more traditional way outside the module system, please let us know [in this issue](https://github.com/facebookincubator/create-react-app/issues/28), and we will consider support for this. ## Adding Bootstrap You don’t have to use [React Bootstrap](https://react-bootstrap.github.io) together with React but it is a popular library for integrating Bootstrap with React apps. If you need it, you can integrate it with Create React App by following these steps: Install React Bootstrap and Bootstrap from NPM. React Bootstrap does not include Bootstrap CSS so this needs to be installed as well: ``` npm install react-bootstrap --save npm install bootstrap@3 --save ``` Import Bootstrap CSS and optionally Bootstrap theme CSS in the ```src/index.js``` file: ```js import 'bootstrap/dist/css/bootstrap.css'; import 'bootstrap/dist/css/bootstrap-theme.css'; ``` Import required React Bootstrap components within ```src/App.js``` file or your custom component files: ```js import { Navbar, Jumbotron, Button } from 'react-bootstrap'; ``` Now you are ready to use the imported React Bootstrap components within your component hierarchy defined in the render method. Here is an example [`App.js`](https://gist.githubusercontent.com/gaearon/85d8c067f6af1e56277c82d19fd4da7b/raw/6158dd991b67284e9fc8d70b9d973efe87659d72/App.js) redone using React Bootstrap. ## Adding Flow Flow typing is currently [not supported out of the box](https://github.com/facebookincubator/create-react-app/issues/72) with the default `.flowconfig` generated by Flow. If you run it, you might get errors like this: ```js node_modules/fbjs/lib/Deferred.js.flow:60 60: Promise.prototype.done.apply(this._promise, arguments); ^^^^ property `done`. Property not found in 495: declare class Promise<+R> { ^ Promise. See lib: /private/tmp/flow/flowlib_34952d31/core.js:495 node_modules/fbjs/lib/shallowEqual.js.flow:29 29: return x !== 0 || 1 / (x: $FlowIssue) === 1 / (y: $FlowIssue); ^^^^^^^^^^ identifier `$FlowIssue`. Could not resolve name src/App.js:3 3: import logo from './logo.svg'; ^^^^^^^^^^^^ ./logo.svg. Required module not found src/App.js:4 4: import './App.css'; ^^^^^^^^^^^ ./App.css. Required module not found src/index.js:5 5: import './index.css'; ^^^^^^^^^^^^^ ./index.css. Required module not found ``` To fix this, change your `.flowconfig` to look like this: ```ini [libs] ./node_modules/fbjs/flow/lib [options] esproposal.class_static_fields=enable esproposal.class_instance_fields=enable module.name_mapper='^\(.*\)\.css$' -> 'react-scripts/config/flow/css' module.name_mapper='^\(.*\)\.\(jpg\|png\|gif\|eot\|otf\|webp\|svg\|ttf\|woff\|woff2\|mp4\|webm\)$' -> 'react-scripts/config/flow/file' suppress_type=$FlowIssue suppress_type=$FlowFixMe ``` Re-run flow, and you shouldn’t get any extra issues. If you later `eject`, you’ll need to replace `react-scripts` references with the `<PROJECT_ROOT>` placeholder, for example: ```ini module.name_mapper='^\(.*\)\.css$' -> '<PROJECT_ROOT>/config/flow/css' module.name_mapper='^\(.*\)\.\(jpg\|png\|gif\|eot\|otf\|webp\|svg\|ttf\|woff\|woff2\|mp4\|webm\)$' -> '<PROJECT_ROOT>/config/flow/file' ``` We will consider integrating more tightly with Flow in the future so that you don’t have to do this. ## Adding Custom Environment Variables >Note: this feature is available with `react-scripts@0.2.3` and higher. Your project can consume variables declared in your environment as if they were declared locally in your JS files. By default you will have `NODE_ENV` defined for you, and any other environment variables starting with `REACT_APP_`. These environment variables will be defined for you on `process.env`. For example, having an environment variable named `REACT_APP_SECRET_CODE` will be exposed in your JS as `process.env.REACT_APP_SECRET_CODE`, in addition to `process.env.NODE_ENV`. These environment variables can be useful for displaying information conditionally based on where the project is deployed or consuming sensitive data that lives outside of version control. First, you need to have environment variables defined, which can vary between OSes. For example, let's say you wanted to consume a secret defined in the environment inside a `<form>`: ```jsx render() { return ( <div> <small>You are running this application in <b>{process.env.NODE_ENV}</b> mode.</small> <form> <input type="hidden" defaultValue={process.env.REACT_APP_SECRET_CODE} /> </form> </div> ); } ``` The above form is looking for a variable called `REACT_APP_SECRET_CODE` from the environment. In order to consume this value, we need to have it defined in the environment: ### Windows (cmd.exe) ```cmd set REACT_APP_SECRET_CODE=abcdef&&npm start ``` (Note: the lack of whitespace is intentional.) ### Linux, OS X (Bash) ```bash REACT_APP_SECRET_CODE=abcdef npm start ``` > Note: Defining environment variables in this manner is temporary for the life of the shell session. Setting permanent environment variables is outside the scope of these docs. With our environment variable defined, we start the app and consume the values. Remember that the `NODE_ENV` variable will be set for you automatically. When you load the app in the browser and inspect the `<input>`, you will see its value set to `abcdef`, and the bold text will show the environment provided when using `npm start`: ```html <div> <small>You are running this application in <b>development</b> mode.</small> <form> <input type="hidden" value="abcdef" /> </form> </div> ``` Having access to the `NODE_ENV` is also useful for performing actions conditionally: ```js if (process.env.NODE_ENV !== 'production') { analytics.disable(); } ``` ## Integrating with a Node Backend Check out [this tutorial](https://www.fullstackreact.com/articles/using-create-react-app-with-a-server/) for instructions on integrating an app with a Node backend running on another port, and using `fetch()` to access it. You can find the companion GitHub repository [here](https://github.com/fullstackreact/food-lookup-demo). ## Proxying API Requests in Development >Note: this feature is available with `react-scripts@0.2.3` and higher. People often serve the front-end React app from the same host and port as their backend implementation. For example, a production setup might look like this after the app is deployed: ``` / - static server returns index.html with React app /todos - static server returns index.html with React app /api/todos - server handles any /api/* requests using the backend implementation ``` Such setup is **not** required. However, if you **do** have a setup like this, it is convenient to write requests like `fetch('/api/todos')` without worrying about redirecting them to another host or port during development. To tell the development server to proxy any unknown requests to your API server in development, add a `proxy` field to your `package.json`, for example: ```js "proxy": "http://localhost:4000", ``` This way, when you `fetch('/api/todos')` in development, the development server will recognize that it’s not a static asset, and will proxy your request to `http://localhost:4000/api/todos` as a fallback. Conveniently, this avoids [CORS issues](http://stackoverflow.com/questions/21854516/understanding-ajax-cors-and-security-considerations) and error messages like this in development: ``` Fetch API cannot load http://localhost:4000/api/todos. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled. ``` Keep in mind that `proxy` only has effect in development (with `npm start`), and it is up to you to ensure that URLs like `/api/todos` point to the right thing in production. You don’t have to use the `/api` prefix. Any unrecognized request will be redirected to the specified `proxy`. Currently the `proxy` option only handles HTTP requests, and it won’t proxy WebSocket connections. If the `proxy` option is **not** flexible enough for you, alternatively you can: * Enable CORS on your server ([here’s how to do it for Express](http://enable-cors.org/server_expressjs.html)). * Use [environment variables](#adding-custom-environment-variables) to inject the right server host and port into your app. ## Deployment By default, Create React App produces a build assuming your app is hosted at the server root. To override this, specify the `homepage` in your `package.json`, for example: ```js "homepage": "http://mywebsite.com/relativepath", ``` This will let Create React App correctly infer the root path to use in the generated HTML file. ### Now See [this example](https://github.com/xkawi/create-react-app-now) for a zero-configuration single-command deployment with [now](https://zeit.co/now). ### Heroku Use the [Heroku Buildpack for Create React App](https://github.com/mars/create-react-app-buildpack). You can find instructions in [Deploying React with Zero Configuration](https://blog.heroku.com/deploying-react-with-zero-configuration). ### Surge Install the Surge CLI if you haven't already by running `npm install -g surge`. Run the `surge` command and log in you or create a new account. You just need to specify the *build* folder and your custom domain, and you are done. ```sh email: email@domain.com password: ******** project path: /path/to/project/build size: 7 files, 1.8 MB domain: create-react-app.surge.sh upload: [====================] 100%, eta: 0.0s propagate on CDN: [====================] 100% plan: Free users: email@domain.com IP Address: X.X.X.X Success! Project is published and running at create-react-app.surge.sh ``` Note that in order to support routers that use html5 `pushState` API, you may want to rename the `index.html` in your build folder to `200.html` before deploying to Surge. This [ensures that every URL falls back to that file](https://surge.sh/help/adding-a-200-page-for-client-side-routing). ### GitHub Pages >Note: this feature is available with `react-scripts@0.2.0` and higher. Open your `package.json` and add a `homepage` field: ```js "homepage": "http://myusername.github.io/my-app", ``` **The above step is important!** Create React App uses the `homepage` field to determine the root URL in the built HTML file. Now, whenever you run `npm run build`, you will see a cheat sheet with a sequence of commands to deploy to GitHub pages: ```sh git commit -am "Save local changes" git checkout -B gh-pages git add -f build git commit -am "Rebuild website" git filter-branch -f --prune-empty --subdirectory-filter build git push -f origin gh-pages git checkout - ``` You may copy and paste them, or put them into a custom shell script. You may also customize them for another hosting provider. Note that GitHub Pages doesn't support routers that use the HTML5 `pushState` history API under the hood (for example, React Router using `browserHistory`). This is because when there is a fresh page load for a url like `http://user.github.io/todomvc/todos/42`, where `/todos/42` is a frontend route, the GitHub Pages server returns 404 because it knows nothing of `/todos/42`. If you want to add a router to a project hosted on GitHub Pages, here are a couple of solutions: * You could switch from using HTML5 history API to routing with hashes. If you use React Router, you can switch to `hashHistory` for this effect, but the URL will be longer and more verbose (for example, `http://user.github.io/todomvc/#/todos/42?_k=yknaj`). [Read more](https://github.com/reactjs/react-router/blob/master/docs/guides/Histories.md#histories) about different history implementations in React Router. * Alternatively, you can use a trick to teach GitHub Pages to handle 404 by redirecting to your `index.html` page with a special redirect parameter. You would need to add a `404.html` file with the redirection code to the `build` folder before deploying your project, and you’ll need to add code handling the redirect parameter to `index.html`. You can find a detailed explanation of this technique [in this guide](https://github.com/rafrex/spa-github-pages). ## Something Missing? If you have ideas for more “How To” recipes that should be on this page, [let us know](https://github.com/facebookincubator/create-react-app/issues) or [contribute some!](https://github.com/facebookincubator/create-react-app/edit/master/template/README.md)
dotnet / Versionsweeper🎯 LTS (or current) versions - GitHub Action that will run as a scheduled CRON job. Ideally, once every few months or as often as necessary to align with .NET version updates.
nyaundid / EC2 AWS AND SHELLSEIS 665 Assignment 2: Linux & Git Overview This week we will focus on becoming familiar with launching a Linux server and working with some basic Linux and Git commands. We will use AWS to launch and host the Linux server. AWS might seem a little confusing at this point. Don’t worry, we will gain much more hands-on experience with AWS throughout the course. The goal is to get you comfortable working with the technology and not overwhelm you with all the details. Requirements You need to have a personal AWS account and GitHub account for this assignment. You should also read the Git Hands-on Guide and Linux Hands-on Guide before beginning this exercise. A word about grading One of the key DevOps practices we learn about in this class is the use of automation to increase the speed and repeatability of processes. Automation is utilized during the assignment grading process to review and assess your work. It’s important that you follow the instructions in each assignment and type in required files and resources with the proper names. All names are case sensitive, so a name like "Web1" is not the same as "web1". If you misspell a name, use the wrong case, or put a file in the wrong directory location you will lose points on your assignment. This is the easiest way to lose points, and also the most preventable. You should always double-check your work to make sure it accurately reflects the requirements specified in the assignment. You should always carefully review the content of your files before submitting your assignment. The assignment Let’s get started! Create GitHub repository The first step in the assignment is to setup a Git repository on GitHub. We will use a special solution called GitHub Classroom for this course which automates the process of setting up student assignment repositories. Here are the basic steps: Click on the following link to open Assignment 2 on the GitHub Classroom site: https://classroom.github.com/a/K4zcVmX- (Links to an external site.)Links to an external site. Click on the Accept this assignment button. GitHub Classroom will provide you with a URL (https) to access the assignment repository. Either copy this address to your clipboard or write it down somewhere. You will need to use this address to set up the repository on a Linux server. Example: https://github.com/UST-SEIS665/hw2-seis665-02-spring2019-<your github id>.git At this point your new repository to ready to use. The repository is currently empty. We will put some content in there soon! Launch Linux server The second step in the assignment is to launch a Linux server using AWS EC2. The server should have the following characteristics: Amazon Linux 2 AMI 64-bit (usually the first option listed) Located in a U.S. region (us-east-1) t2.micro instance type All default instance settings (storage, vpm, security group, etc.) I’ve shown you how to launch EC2 instances in class. You can review it on Canvas. Once you launch the new server, it may take a few minutes to provision. Log into server The next step is to log into the Linux server using a terminal program with a secure shell (SSH) support. You can use iTerm2 (Links to an external site.)Links to an external site. on a Mac and GitBash/PuTTY (Links to an external site.)Links to an external site. on a PC. You will need to have the private server key and the public IP address before attempting to log into the server. The server key is basically your password. If you lose it, you will need to terminate the existing instance and launch a new server. I recommend reusing the same key when launching new servers throughout the class. Note, I make this recommendation to make the learning process easier and not because it is a common security practice. I’ve shown you how to use a terminal application to log into the instance using a Windows desktop. Your personal computer or lab computer may be running a different OS version, but the process is still very similar. You can review the videos on the Canvas. Working with Linux If you’ve made it this far, congratulations! You’ve made it over the toughest hurdle. By the end of this course, I promise you will be able to launch and log into servers in your sleep. You should be looking at a login screen that looks something like this: Last login: Mon Mar 21 21:17:54 2016 from 174-20-199-194.mpls.qwest.net __| __|_ ) _| ( / Amazon Linux AMI ___|\___|___| https://aws.amazon.com/amazon-linux-ami/2015.09-release-notes/ 8 package(s) needed for security, out of 17 available Run "sudo yum update" to apply all updates. ec2-user@ip-172-31-15-26 ~]$ Your terminal cursor is sitting at the shell prompt, waiting for you to type in your first command. Remember the shell? It is a really cool program that lets you start other programs and manage services on the Linux system. The rest of this assignment will be spent working with the shell. Note, when you are asked to type in a command in the steps below, don’t type in the dollar-sign ($) character. This is just meant to represent the command prompt. The actual commands are represented by the characters to the right of the command prompt. Let’s start by asking the shell for some help. Type in: $ help The shell provides you with a list of commands you can run along with possible command options. Next, check out one of the pages in the built-in manual: $ man ls A man page will appear with information on how to use the ls command. This command is used to list the contents of file directories. Either space through the contents of the man page or hit q to exit. Most of the core Linux commands have man pages available. But honestly, some of these man pages are a bit hard to understand. Sometimes your best bet is to search on Google if you are trying to figure out how to use a specific command. When you initially log into Linux, the system places you in your home directory. Each user on the system has a separate home directory. Let’s see where your home directory is located: $ pwd The response should be /home/ec2-user. The pwd command is handy to remember if you ever forget what file directory you are currently located in. If you recall from the Linux Hands-on Guide, this directory is also your current working directory. Type in: $ cd / The cd command let’s you change to a new working directory on the server. In this case, we changed to the root (/) directory. This is the parent of all the other directories on the file system. Type in: $ ls The ls command lists the contents of the current directory. As you can see, root directory contains many other directories. You will become familiar with these directories over time. The ls command provides a very basic directory listing. You need to supply the command with some options if you want to see more detailed information. Type in: $ ls -la See how this command provides you with much more detailed information about the files and directories? You can use this detailed listing to see the owner, group, and access control list settings for each file or directory. Do you see any files listed? Remember, the first character in the access control list column denotes whether a listed item is a file or a directory. You probably see a couple files with names like .autofsck. How come you didn’t see this file when you typed in the lscommand without any options? (Try to run this command again to convince yourself.) Files names that start with a period are called hidden files. These files won’t appear on normal directory listings. Type in: $ cd /var Then, type in: $ ls You will see a directory listing for the /var directory. Next, type in: $ ls .. Huh. This directory listing looks the same as the earlier root directory listing. When you use two periods (..) in a directory path that means you are referring to the parent directory of the current directory. Just think of the two dots as meaning the directory above the current directory. Now, type in: $ cd ~ $ pwd Whoa. We’re back at our home directory again. The tilde character (~) is another one of those handy little directory path shortcuts. It always refers to our personal home directory. Keep in mind that since every user has their own home directory, the tilde shortcut will refer to a unique directory for each logged-in user. Most students are used to navigating a file system by clicking a mouse in nested graphical folders. When they start using a command-line to navigate a file system, they sometimes get confused and lose track of their current position in the file system. Remember, you can always use the pwd command to quickly figure out what directory you are currently working in. Let’s make some changes to the file system. We can easily make our own directories on the file system. Type: mkdir test Now type: ls Cool, there’s our new test directory. Let’s pretend we don’t like that directory name and delete it. Type: rmdir test Now it’s gone. How can you be sure? You should know how to check to see if the directory still exists at this point. Go ahead and check. Let’s create another directory. Type in: $ mkdir documents Next, change to the new directory: $ cd documents Did you notice that your command prompt displays the name of the current directory? Something like: [ec2-user@ip-172-31-15-26 documents]$. Pretty handy, huh? Okay, let’s create our first file in the documents directory. This is just an empty file for training purposes. Type in: $ touch paper.txt Check to see that the new file is in the directory. Now, go back to the previous directory. Remember the double dot shortcut? $ cd .. Okay, we don’t like our documents directory any more. Let’s blow it away. Type in: $ rmdir documents Uh oh. The shell didn’t like that command because the directory isn’t empty. Let’s change back into the documents directory. But this time don’t type in the full name of the directory. You can let shell auto-completion do the typing for you. Type in the first couple characters of the directory name and then hit the tab key: $ cd doc<tab> You should use the tab auto-completion feature often. It saves typing and makes working with the Linux file system much much easier. Tab is your friend. Now, remove the file by typing: $ rm paper.txt Did you try to use the tab key instead of typing in the whole file name? Check to make sure the file was deleted from the directory. Next, create a new file: $ touch file1 We like file1 so much that we want to make a backup copy. Type: $ cp file1 file1-backup Check to make sure the new backup copy was created. We don’t really like the name of that new file, so let’s rename it. Type: $ mv file1-backup backup Moving a file to the same directory and giving it a new name is basically the same thing as renaming it. We could have moved it to a different directory if we wanted. Let’s list all of the files in the current directory that start with the letter f: $ ls f* Using wildcard pattern matching in file commands is really useful if you want the command to impact or filter a group of files. Now, go up one directory to the parent directory (remember the double dot shortcut?) We tried to remove the documents directory earlier when it had files in it. Obviously that won’t work again. However, we can use a more powerful command to destroy the directory and vanquish its contents. Behold, the all powerful remove command: $ rm -fr documents Did you remember to use auto-completion when typing in documents? This command and set of options forcibly removes the directory and its contents. It’s a dangerous command wielded by the mightiest Linux wizards. Okay, maybe that’s a bit of an exaggeration. Just be careful with it. Check to make sure the documents directory is gone before proceeding. Let’s continue. Change to the directory /var and make a directory called test. Ugh. Permission denied. We created this darn Linux server and we paid for it. Shouldn’t we be able to do anything we want on it? You logged into the system as a user called ec2-user. While this user can create and manage files in its home directory, it cannot change files all across the system. At least it can’t as a normal user. The ec2-user is a member of the root group, so it can escalate its privileges to super-user status when necessary. Let’s try it: $ sudo mkdir test Check to make sure the directory exists now. Using sudo we can execute commands as a super-user. We can do anything we want now that we know this powerful new command. Go ahead and delete the test directory. Did you remember to use sudo before the rmdir command? Check to make sure the directory is gone. You might be asking yourself the question: why can we list the contents of the /var directory but not make changes? That’s because all users have read access to the /var directory and the ls command is a read function. Only the root users or those acting as a super-user can write changes to the directory. Let’s go back to our home directory: $ cd ~ Editing text files is a really common task on Linux systems because many of the application configuration files are text files. We can create a text file by using a text editor. Type in: $ nano myfile.conf The shell starts up the nano text editor and places your terminal cursor in the editing screen. Nano is a simple text-based word processor. Type in a few lines of text. When you’re done writing your novel, hit ctrl-x and answer y to the prompt to save your work. Finally, hit enter to save the text to the filename you specified. Check to see that your file was saved in the directory. You can take a look at the contents of your file by typing: $ cat myfile.conf The cat command displays your text file content on the terminal screen. This command works fine for displaying small text files. But if your file is hundreds of lines long, the content will scroll down your terminal screen so fast that you won’t be able to easily read it. There’s a better way to view larger text files. Type in: $ less myfile.conf The less command will page the display of a text file, allowing you to page through the contents of the file using the space bar. Your text file is probably too short to see the paging in action though. Hit q to quit out of the less text viewer. Hit the up-arrow key on your keyboard a few times until the commmand nano myfile.conf appears next to your command prompt. Cool, huh? The up-arrow key allows you to replay a previously run command. Linux maintains a list of all the commands you have run since you logged into the server. This is called the command history. It’s a really useful feature if you have to re-run a complex command again. Now, hit ctrl-c. This cancels whatever command is displayed on the command line. Type in the following command to create a couple empty files in the directory: $ touch file1 file2 file3 Confirm that the files were created. Some commands, like touch. allow you to specify multiple files as arguments. You will find that Linux commands have all kinds of ways to make tasks more efficient like this. Throughout this assignment, we have been running commands and viewing results on the terminal screen. The screen is the standard place for commands to output results. It’s known as the standard out (stdout). However, it’s really useful to output results to the file system sometimes. Type in: $ ls > listing.txt Take a look at the directory listing now. You just created a new file. View the contents of the listing.txt file. What do you see? Instead of sending the output from the ls command to the screen we sent it to a text file. Let’s try another one. Type: $ cat myfile.conf > listing.txt Take a look at the contents of the listing.txt file again. It looks like your myfile.conf file now. It’s like you made a copy of it. But what happened to the previous content in the listing.txt file? When you redirect the output of a command using the right angle-bracket character (>), the output overwrites the existing file. Type this command in: $ cat myfile.conf >> listing.txt Now look at the contents of the listing.txt file. You should see your original content displayed twice. When you use two angle-bracket characters in the commmand the output appends (or adds to) the file instead of overwriting it. We redirected the output from a command to a text file. It’s also possible to redirect the input to a command. Typically we use a keyboard to provide input, but sometimes it makes more sense to input a file to a command. For example, how many words are in your new listing.txt file? Let’s find out. Type in: $ wc -w < listing.txt Did you get a number? This command inputs the listing.txt file into a word count program called wc. Type in the command: $ ls /usr/bin The terminal screen probably scrolled quickly as filenames flashed by. The /usr/bin directory holds quite a few files. It would be nice if we could page through the contents of this directory. Well, we can. We can use a special shell feature called pipes. In previous steps, we redirected I/O using the file system. Pipes allow us to redirect I/O between programs. We can redirect the output from one program into another. Type in: $ ls /usr/bin | less Now the directory listing is paged. Hit the spacebar to page through the listing. The pipe, represented by a vertical bar character (|), takes the output from the ls command and redirects it to the less command where the resulting output is paged. Pipes are super powerful and used all the time by savvy Linux operators. Hit the q key to quit the paginated directory listing command. Working with shell scripts Now things are going to get interesting. We’ve been manually typing in commands throughout this exercise. If we were running a set of repetitive tasks, we would want to automate the process as much as possible. The shell makes it really easy to automate tasks using shell scripts. The shell provides many of the same features as a basic procedural programming language. Let’s write some code. Type in this command: $ j=123 $ echo $j We just created a variable named j referencing the string 123. The echo command printed out the value of the variable. We had to use a dollar sign ($) when referencing the variable in another command. Next, type in: $ j=1+1 $ echo $j Is that what you expected? The shell just interprets the variable value as a string. It’s not going to do any sort of computation. Typing in shell script commands on the command line is sort of pointless. We want to be able to create scripts that we can run over-and-over. Let’s create our first shell script. Use the nano editor to create a file named myscript. When the file is open in the editor, type in the following lines of code: #!/bin/bash echo Hello $1 Now quit the editor and save your file. We can run our script by typing: $ ./myscript World Er, what happened? Permission denied. Didn’t we create this file? Why can’t we run it? We can’t run the script file because we haven’t set the execute permission on the file. Type in: $ chmod u+x myscript This modifies the file access control list to allow the owner of the file to execute it. Let’s try to run the command again. Hit the up-arrow key a couple times until the ./myscript World command is displayed and hit enter. Hooray! Our first shell script. It’s probably a bit underwhelming. No problem, we’ll make it a little more complex. The script took a single argument called World. Any arguments provided to a shell script are represented as consecutively numbered variables inside the script ($1, $2, etc). Pretty simple. You might be wondering why we had to type the ./ characters before the name of our script file. Try to type in the command without them: $ myscript World Command not found. That seems a little weird. Aren’t we currently in the directory where the shell script is located? Well, that’s just not how the shell works. When you enter a command into the shell, it looks for the command in a predefined set of directories on the server called your PATH. Since your script file isn’t in your special path, the shell reports it as not found. By typing in the ./ characters before the command name you are basically forcing the shell to look for your script in the current directory instead of the default path. Create another file called cleanup using nano. In the file editor window type: #!/bin/bash # My cleanup script mkdir archive mv file* archive Exit the editor window and save the file. Change the permissions on the script file so that you can execute it. Now run the command: $ ./cleanup Take a look at the file directory listing. Notice the archive directory? List the contents of that directory. The script automatically created a new directory and moved three files into it. Anything you can do manually at a command prompt can be automated using a shell script. Let’s create one more shell script. Use nano to create a script called namelist. Here is the content of the script: #!/bin/bash # for-loop test script names='Jason John Jane' for i in $names do echo Hello $i done Change the permissions on the script file so that you can execute it. Run the command: $ ./namelist The script will loop through a set of names stored in a variable displaying each one. Scripts support several programming constructs like for-loops, do-while loops, and if-then-else. These building blocks allow you to create fairly complex scripts for automating tasks. Installing packages and services We’re nearing the end of this assignment. But before we finish, let’s install some new software packages on our server. The first thing we should do is make sure all the current packages installed on our Linux server are up-to-date. Type in: $ sudo yum update -y This is one of those really powerful commands that requires sudo access. The system will review the currently installed packages and go out to the Internet and download appropriate updates. Next, let’s install an Apache web server on our system. Type in: $ sudo yum install httpd -y Bam! You probably never knew that installing a web server was so easy. We’re not going to actually use the web server in this exercise, but we will in future assignments. We installed the web server, but is it actually running? Let’s check. Type in: $ sudo service httpd status Nope. Let’s start it. Type: $ sudo service httpd start We can use the service command to control the services running on the system. Let’s setup the service so that it automatically starts when the system boots up. Type in: $ sudo chkconfig httpd on Cool. We installed the Apache web server on our system, but what other programs are currently running? We can use the pscommand to find out. Type in: $ ps -ax Lots of processes are running on our system. We can even look at the overall performance of our system using the topcommand. Let’s try that now. Type in: $ top The display might seem a little overwhelming at first. You should see lots of performance information displayed including the cpu usage, free memory, and a list of running tasks. We’re almost across the finish line. Let’s make sure all of our valuable work is stored in a git repository. First, we need to install git. Type in the command: $ sudo yum install git -y Check your work It’s very important to check your work before submitting it for grading. A misspelled, misplaced or missing file will cost you points. This may seem harsh, but the reality is that these sorts of mistakes have consequences in the real world. For example, a server instance could fail to launch properly and impact customers because a single required file is missing. Here is what the contents of your git repository should look like before final submission: ┣archive ┃ ┣ file1 ┃ ┣ file2 ┃ ┗ file3 ┣ namelist ┗ myfile.conf Saving our work in the git repository Next, make sure you are still in your home directory (/home/ec2-user). We will install the git repository you created at the beginning of this exercise. You will need to modify this command by typing in the GitHub repository URL you copied earlier. $ git clone <your GitHub URL here>.git Example: git clone https://github.com/UST-SEIS665/hw2-seis665-02-spring2019-<your github id>.git The git application will ask you for your GitHub username and password. Note, if you have multi-factor authentication enabled on your GitHub account you will need to provide a personal token instead of your password. Git will clone (copy) the repository from GitHub to your Linux server. Since the repository is empty the clone happens almost instantly. Check to make sure that a sub-directory called "hw2-seis665-02-spring2019-<username>" exists in the current directory (where <username> is your GitHub account name). Git automatically created this directory as part of the cloning process. Change to the hw2-seis665-02-spring2019-<username> directory and type: $ ls -la Notice the .git hidden directory? This is where git actually stores all of the file changes in your repository. Nothing is actually in your repository yet. Change back to the parent directory (cd ..). Next, let’s move some of our files into the repository. Type: $ mv archive hw2-seis665-02-spring2019-<username> $ mv namelist hw2-seis665-02-spring2019-<username> $ mv myfile.conf hw2-seis665-02-spring2019-<username> Hopefully, you remembered to use the auto-complete function to reduce some of that typing. Change to the hw2-seis665-02-spring2019-<username> directory and list the directory contents. Your files are in the working directory, but are not actually stored in the repository because they haven’t been committed yet. Type in: $ git status You should see a list of untracked files. Let’s tell git that we want these files tracked. Type in: $ git add * Now type in the git status command again. Notice how all the files are now being tracked and are ready to be committed. These files are in the git staging area. We’ll commit them to the repository next. Type: $ git commit -m 'assignment 2 files' Next, take a look at the commit log. Type: $ git log You should see your commit listed along with an assigned hash (long string of random-looking characters). Finally, let’s save the repository to our GitHub account. Type in: $ git push origin master The git client will ask you for your GitHub username and password before pushing the repository. Go back to the GitHub.com website and login if you have been logged out. Click on the repository link for the assignment. Do you see your files listed there? Congratulations, you completed the exercise! Terminate server The last step is to terminate your Linux instance. AWS will bill you for every hour the instance is running. The cost is nominal, but there’s no need to rack up unnecessary charges. Here are the steps to terminate your instance: Log into your AWS account and click on the EC2 dashboard. Click the Instances menu item. Select your server in the instances table. Click on the Actions drop down menu above the instances table. Select the Instance State menu option Click on the Terminate action. Your Linux instance will shutdown and disappear in a few minutes. The EC2 dashboard will continue to display the instance on your instance listing for another day or so. However, the state of the instance will be terminated. Submitting your assignment — IMPORTANT! If you haven’t already, please e-mail me your GitHub username in order to receive credit for this assignment. There is no need to email me to tell me that you have committed your work to GitHub or to ask me if your GitHub submission worked. If you can see your work in your GitHub repository, I can see your work.
SystemJargon / FiltersA repo containing DNS blocklists and allowlist/s. Various categories of lists, updated and validated often. Wiki included.
arashstar1 / Bot LuaCode Issues 0 Pull requests 0 Pulse MaTaDoR/ 3233fdf V 5.7 MaTaDoR @MaTaDoRTeaMMaTaDoRTeaM committed on GitHub about 1 month ago 2 changed files 2,704 additions and 0 deletions cli/tg/tdcli.lua @@ -0,0 +1,2704 @@ +--[[ + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, + MA 02110-1301, USA. + +]]-- + +-- Vector example form is like this: {[0] = v} or {v1, v2, v3, [0] = v} +-- If false or true crashed your telegram-cli, try to change true to 1 and false to 0 + +-- Main Bot Framework +local M = {} + +-- @chat_id = user, group, channel, and broadcast +-- @group_id = normal group +-- @channel_id = channel and broadcast +local function getChatId(chat_id) + local chat = {} + local chat_id = tostring(chat_id) + + if chat_id:match('^-100') then + local channel_id = chat_id:gsub('-100', '') + chat = {ID = channel_id, type = 'channel'} + else + local group_id = chat_id:gsub('-', '') + chat = {ID = group_id, type = 'group'} + end + + return chat +end + +local function getInputFile(file) + if file:match('/') then + infile = {ID = "InputFileLocal", path_ = file} + elseif file:match('^%d+$') then + infile = {ID = "InputFileId", id_ = file} + else + infile = {ID = "InputFilePersistentId", persistent_id_ = file} + end + + return infile +end + +-- User can send bold, italic, and monospace text uses HTML or Markdown format. +local function getParseMode(parse_mode) + if parse_mode then + local mode = parse_mode:lower() + + if mode == 'markdown' or mode == 'md' then + P = {ID = "TextParseModeMarkdown"} + elseif mode == 'html' then + P = {ID = "TextParseModeHTML"} + end + end + + return P +end + +-- Returns current authorization state, offline request +local function getAuthState(dl_cb, cmd) + tdcli_function ({ + ID = "GetAuthState", + }, dl_cb, cmd) +end + +M.getAuthState = getAuthState + +-- Sets user's phone number and sends authentication code to the user. +-- Works only when authGetState returns authStateWaitPhoneNumber. +-- If phone number is not recognized or another error has happened, returns an error. Otherwise returns authStateWaitCode +-- @phone_number User's phone number in any reasonable format +-- @allow_flash_call Pass True, if code can be sent via flash call to the specified phone number +-- @is_current_phone_number Pass true, if the phone number is used on the current device. Ignored if allow_flash_call is False +local function setAuthPhoneNumber(phone_number, allow_flash_call, is_current_phone_number, dl_cb, cmd) + tdcli_function ({ + ID = "SetAuthPhoneNumber", + phone_number_ = phone_number, + allow_flash_call_ = allow_flash_call, + is_current_phone_number_ = is_current_phone_number + }, dl_cb, cmd) +end + +M.setAuthPhoneNumber = setAuthPhoneNumber + +-- Resends authentication code to the user. +-- Works only when authGetState returns authStateWaitCode and next_code_type of result is not null. +-- Returns authStateWaitCode on success +local function resendAuthCode(dl_cb, cmd) + tdcli_function ({ + ID = "ResendAuthCode", + }, dl_cb, cmd) +end + +M.resendAuthCode = resendAuthCode + +-- Checks authentication code. +-- Works only when authGetState returns authStateWaitCode. +-- Returns authStateWaitPassword or authStateOk on success +-- @code Verification code from SMS, Telegram message, voice call or flash call +-- @first_name User first name, if user is yet not registered, 1-255 characters +-- @last_name Optional user last name, if user is yet not registered, 0-255 characters +local function checkAuthCode(code, first_name, last_name, dl_cb, cmd) + tdcli_function ({ + ID = "CheckAuthCode", + code_ = code, + first_name_ = first_name, + last_name_ = last_name + }, dl_cb, cmd) +end + +M.checkAuthCode = checkAuthCode + +-- Checks password for correctness. +-- Works only when authGetState returns authStateWaitPassword. +-- Returns authStateOk on success +-- @password Password to check +local function checkAuthPassword(password, dl_cb, cmd) + tdcli_function ({ + ID = "CheckAuthPassword", + password_ = password + }, dl_cb, cmd) +end + +M.checkAuthPassword = checkAuthPassword + +-- Requests to send password recovery code to email. +-- Works only when authGetState returns authStateWaitPassword. +-- Returns authStateWaitPassword on success +local function requestAuthPasswordRecovery(dl_cb, cmd) + tdcli_function ({ + ID = "RequestAuthPasswordRecovery", + }, dl_cb, cmd) +end + +M.requestAuthPasswordRecovery = requestAuthPasswordRecovery + +-- Recovers password with recovery code sent to email. +-- Works only when authGetState returns authStateWaitPassword. +-- Returns authStateOk on success +-- @recovery_code Recovery code to check +local function recoverAuthPassword(recovery_code, dl_cb, cmd) + tdcli_function ({ + ID = "RecoverAuthPassword", + recovery_code_ = recovery_code + }, dl_cb, cmd) +end + +M.recoverAuthPassword = recoverAuthPassword + +-- Logs out user. +-- If force == false, begins to perform soft log out, returns authStateLoggingOut after completion. +-- If force == true then succeeds almost immediately without cleaning anything at the server, but returns error with code 401 and description "Unauthorized" +-- @force If true, just delete all local data. Session will remain in list of active sessions +local function resetAuth(force, dl_cb, cmd) + tdcli_function ({ + ID = "ResetAuth", + force_ = force or nil + }, dl_cb, cmd) +end + +M.resetAuth = resetAuth + +-- Check bot's authentication token to log in as a bot. +-- Works only when authGetState returns authStateWaitPhoneNumber. +-- Can be used instead of setAuthPhoneNumber and checkAuthCode to log in. +-- Returns authStateOk on success +-- @token Bot token +local function checkAuthBotToken(token, dl_cb, cmd) + tdcli_function ({ + ID = "CheckAuthBotToken", + token_ = token + }, dl_cb, cmd) +end + +M.checkAuthBotToken = checkAuthBotToken + +-- Returns current state of two-step verification +local function getPasswordState(dl_cb, cmd) + tdcli_function ({ + ID = "GetPasswordState", + }, dl_cb, cmd) +end + +M.getPasswordState = getPasswordState + +-- Changes user password. +-- If new recovery email is specified, then error EMAIL_UNCONFIRMED is returned and password change will not be applied until email confirmation. +-- Application should call getPasswordState from time to time to check if email is already confirmed +-- @old_password Old user password +-- @new_password New user password, may be empty to remove the password +-- @new_hint New password hint, can be empty +-- @set_recovery_email Pass True, if recovery email should be changed +-- @new_recovery_email New recovery email, may be empty +local function setPassword(old_password, new_password, new_hint, set_recovery_email, new_recovery_email, dl_cb, cmd) + tdcli_function ({ + ID = "SetPassword", + old_password_ = old_password, + new_password_ = new_password, + new_hint_ = new_hint, + set_recovery_email_ = set_recovery_email, + new_recovery_email_ = new_recovery_email + }, dl_cb, cmd) +end + +M.setPassword = setPassword + +-- Returns set up recovery email. +-- This method can be used to verify a password provided by the user +-- @password Current user password +local function getRecoveryEmail(password, dl_cb, cmd) + tdcli_function ({ + ID = "GetRecoveryEmail", + password_ = password + }, dl_cb, cmd) +end + +M.getRecoveryEmail = getRecoveryEmail + +-- Changes user recovery email. +-- If new recovery email is specified, then error EMAIL_UNCONFIRMED is returned and email will not be changed until email confirmation. +-- Application should call getPasswordState from time to time to check if email is already confirmed. +-- If new_recovery_email coincides with the current set up email succeeds immediately and aborts all other requests waiting for email confirmation +-- @password Current user password +-- @new_recovery_email New recovery email +local function setRecoveryEmail(password, new_recovery_email, dl_cb, cmd) + tdcli_function ({ + ID = "SetRecoveryEmail", + password_ = password, + new_recovery_email_ = new_recovery_email + }, dl_cb, cmd) +end + +M.setRecoveryEmail = setRecoveryEmail + +-- Requests to send password recovery code to email +local function requestPasswordRecovery(dl_cb, cmd) + tdcli_function ({ + ID = "RequestPasswordRecovery", + }, dl_cb, cmd) +end + +M.requestPasswordRecovery = requestPasswordRecovery + +-- Recovers password with recovery code sent to email +-- @recovery_code Recovery code to check +local function recoverPassword(recovery_code, dl_cb, cmd) + tdcli_function ({ + ID = "RecoverPassword", + recovery_code_ = tostring(recovery_code) + }, dl_cb, cmd) +end + +M.recoverPassword = recoverPassword + +-- Returns current logged in user +local function getMe(dl_cb, cmd) + tdcli_function ({ + ID = "GetMe", + }, dl_cb, cmd) +end + +M.getMe = getMe + +-- Returns information about a user by its identifier, offline request if current user is not a bot +-- @user_id User identifier +local function getUser(user_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetUser", + user_id_ = user_id + }, dl_cb, cmd) +end + +M.getUser = getUser + +-- Returns full information about a user by its identifier +-- @user_id User identifier +local function getUserFull(user_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetUserFull", + user_id_ = user_id + }, dl_cb, cmd) +end + +M.getUserFull = getUserFull + +-- Returns information about a group by its identifier, offline request if current user is not a bot +-- @group_id Group identifier +local function getGroup(group_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetGroup", + group_id_ = getChatId(group_id).ID + }, dl_cb, cmd) +end + +M.getGroup = getGroup + +-- Returns full information about a group by its identifier +-- @group_id Group identifier +local function getGroupFull(group_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetGroupFull", + group_id_ = getChatId(group_id).ID + }, dl_cb, cmd) +end + +M.getGroupFull = getGroupFull + +-- Returns information about a channel by its identifier, offline request if current user is not a bot +-- @channel_id Channel identifier +local function getChannel(channel_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetChannel", + channel_id_ = getChatId(channel_id).ID + }, dl_cb, cmd) +end + +M.getChannel = getChannel + +-- Returns full information about a channel by its identifier, cached for at most 1 minute +-- @channel_id Channel identifier +local function getChannelFull(channel_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetChannelFull", + channel_id_ = getChatId(channel_id).ID + }, dl_cb, cmd) +end + +M.getChannelFull = getChannelFull + +-- Returns information about a secret chat by its identifier, offline request +-- @secret_chat_id Secret chat identifier +local function getSecretChat(secret_chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetSecretChat", + secret_chat_id_ = secret_chat_id + }, dl_cb, cmd) +end + +M.getSecretChat = getSecretChat + +-- Returns information about a chat by its identifier, offline request if current user is not a bot +-- @chat_id Chat identifier +local function getChat(chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetChat", + chat_id_ = chat_id + }, dl_cb, cmd) +end + +M.getChat = getChat + +-- Returns information about a message +-- @chat_id Identifier of the chat, message belongs to +-- @message_id Identifier of the message to get +local function getMessage(chat_id, message_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetMessage", + chat_id_ = chat_id, + message_id_ = message_id + }, dl_cb, cmd) +end + +M.getMessage = getMessage + +-- Returns information about messages. +-- If message is not found, returns null on the corresponding position of the result +-- @chat_id Identifier of the chat, messages belongs to +-- @message_ids Identifiers of the messages to get +local function getMessages(chat_id, message_ids, dl_cb, cmd) + tdcli_function ({ + ID = "GetMessages", + chat_id_ = chat_id, + message_ids_ = message_ids -- vector + }, dl_cb, cmd) +end + +M.getMessages = getMessages + +-- Returns information about a file, offline request +-- @file_id Identifier of the file to get +local function getFile(file_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetFile", + file_id_ = file_id + }, dl_cb, cmd) +end + +M.getFile = getFile + +-- Returns information about a file by its persistent id, offline request +-- @persistent_file_id Persistent identifier of the file to get +local function getFilePersistent(persistent_file_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetFilePersistent", + persistent_file_id_ = persistent_file_id + }, dl_cb, cmd) +end + +M.getFilePersistent = getFilePersistent + +-- Returns list of chats in the right order, chats are sorted by (order, chat_id) in decreasing order. +-- For example, to get list of chats from the beginning, the offset_order should be equal 2^63 - 1 +-- @offset_order Chat order to return chats from +-- @offset_chat_id Chat identifier to return chats from +-- @limit Maximum number of chats to be returned +local function getChats(offset_order, offset_chat_id, limit, dl_cb, cmd) + if not limit or limit > 20 then + limit = 20 + end + + tdcli_function ({ + ID = "GetChats", + offset_order_ = offset_order or 9223372036854775807, + offset_chat_id_ = offset_chat_id or 0, + limit_ = limit + }, dl_cb, cmd) +end + +M.getChats = getChats + +-- Searches public chat by its username. +-- Currently only private and channel chats can be public. +-- Returns chat if found, otherwise some error is returned +-- @username Username to be resolved +local function searchPublicChat(username, dl_cb, cmd) + tdcli_function ({ + ID = "SearchPublicChat", + username_ = username + }, dl_cb, cmd) +end + +M.searchPublicChat = searchPublicChat + +-- Searches public chats by prefix of their username. +-- Currently only private and channel (including supergroup) chats can be public. +-- Returns meaningful number of results. +-- Returns nothing if length of the searched username prefix is less than 5. +-- Excludes private chats with contacts from the results +-- @username_prefix Prefix of the username to search +local function searchPublicChats(username_prefix, dl_cb, cmd) + tdcli_function ({ + ID = "SearchPublicChats", + username_prefix_ = username_prefix + }, dl_cb, cmd) +end + +M.searchPublicChats = searchPublicChats + +-- Searches for specified query in the title and username of known chats, offline request. +-- Returns chats in the order of them in the chat list +-- @query Query to search for, if query is empty, returns up to 20 recently found chats +-- @limit Maximum number of chats to be returned +local function searchChats(query, limit, dl_cb, cmd) + if not limit or limit > 20 then + limit = 20 + end + + tdcli_function ({ + ID = "SearchChats", + query_ = query, + limit_ = limit + }, dl_cb, cmd) +end + +M.searchChats = searchChats + +-- Adds chat to the list of recently found chats. +-- The chat is added to the beginning of the list. +-- If the chat is already in the list, at first it is removed from the list +-- @chat_id Identifier of the chat to add +local function addRecentlyFoundChat(chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "AddRecentlyFoundChat", + chat_id_ = chat_id + }, dl_cb, cmd) +end + +M.addRecentlyFoundChat = addRecentlyFoundChat + +-- Deletes chat from the list of recently found chats +-- @chat_id Identifier of the chat to delete +local function deleteRecentlyFoundChat(chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteRecentlyFoundChat", + chat_id_ = chat_id + }, dl_cb, cmd) +end + +M.deleteRecentlyFoundChat = deleteRecentlyFoundChat + +-- Clears list of recently found chats +local function deleteRecentlyFoundChats(dl_cb, cmd) + tdcli_function ({ + ID = "DeleteRecentlyFoundChats", + }, dl_cb, cmd) +end + +M.deleteRecentlyFoundChats = deleteRecentlyFoundChats + +-- Returns list of common chats with an other given user. +-- Chats are sorted by their type and creation date +-- @user_id User identifier +-- @offset_chat_id Chat identifier to return chats from, use 0 for the first request +-- @limit Maximum number of chats to be returned, up to 100 +local function getCommonChats(user_id, offset_chat_id, limit, dl_cb, cmd) + if not limit or limit > 100 then + limit = 100 + end + + tdcli_function ({ + ID = "GetCommonChats", + user_id_ = user_id, + offset_chat_id_ = offset_chat_id, + limit_ = limit + }, dl_cb, cmd) +end + +M.getCommonChats = getCommonChats + +-- Returns messages in a chat. +-- Automatically calls openChat. +-- Returns result in reverse chronological order, i.e. in order of decreasing message.message_id +-- @chat_id Chat identifier +-- @from_message_id Identifier of the message near which we need a history, you can use 0 to get results from the beginning, i.e. from oldest to newest +-- @offset Specify 0 to get results exactly from from_message_id or negative offset to get specified message and some newer messages +-- @limit Maximum number of messages to be returned, should be positive and can't be greater than 100. +-- If offset is negative, limit must be greater than -offset. +-- There may be less than limit messages returned even the end of the history is not reached +local function getChatHistory(chat_id, from_message_id, offset, limit, dl_cb, cmd) + if not limit or limit > 100 then + limit = 100 + end + + tdcli_function ({ + ID = "GetChatHistory", + chat_id_ = chat_id, + from_message_id_ = from_message_id, + offset_ = offset or 0, + limit_ = limit + }, dl_cb, cmd) +end + +M.getChatHistory = getChatHistory + +-- Deletes all messages in the chat. +-- Can't be used for channel chats +-- @chat_id Chat identifier +-- @remove_from_chat_list Pass true, if chat should be removed from the chat list +local function deleteChatHistory(chat_id, remove_from_chat_list, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteChatHistory", + chat_id_ = chat_id, + remove_from_chat_list_ = remove_from_chat_list + }, dl_cb, cmd) +end + +M.deleteChatHistory = deleteChatHistory + +-- Searches for messages with given words in the chat. +-- Returns result in reverse chronological order, i. e. in order of decreasimg message_id. +-- Doesn't work in secret chats +-- @chat_id Chat identifier to search in +-- @query Query to search for +-- @from_message_id Identifier of the message from which we need a history, you can use 0 to get results from beginning +-- @limit Maximum number of messages to be returned, can't be greater than 100 +-- @filter Filter for content of searched messages +-- filter = Empty|Animation|Audio|Document|Photo|Video|Voice|PhotoAndVideo|Url|ChatPhoto +local function searchChatMessages(chat_id, query, from_message_id, limit, filter, dl_cb, cmd) + if not limit or limit > 100 then + limit = 100 + end + + tdcli_function ({ + ID = "SearchChatMessages", + chat_id_ = chat_id, + query_ = query, + from_message_id_ = from_message_id, + limit_ = limit, + filter_ = { + ID = 'SearchMessagesFilter' .. filter + }, + }, dl_cb, cmd) +end + +M.searchChatMessages = searchChatMessages + +-- Searches for messages in all chats except secret chats. Returns result in reverse chronological order, i. e. in order of decreasing (date, chat_id, message_id) +-- @query Query to search for +-- @offset_date Date of the message to search from, you can use 0 or any date in the future to get results from the beginning +-- @offset_chat_id Chat identifier of the last found message or 0 for the first request +-- @offset_message_id Message identifier of the last found message or 0 for the first request +-- @limit Maximum number of messages to be returned, can't be greater than 100 +local function searchMessages(query, offset_date, offset_chat_id, offset_message_id, limit, dl_cb, cmd) + if not limit or limit > 100 then + limit = 100 + end + + tdcli_function ({ + ID = "SearchMessages", + query_ = query, + offset_date_ = offset_date, + offset_chat_id_ = offset_chat_id, + offset_message_id_ = offset_message_id, + limit_ = limit + }, dl_cb, cmd) +end + +M.searchMessages = searchMessages + +-- Invites bot to a chat (if it is not in the chat) and send /start to it. +-- Bot can't be invited to a private chat other than chat with the bot. +-- Bots can't be invited to broadcast channel chats and secret chats. +-- Returns sent message. +-- UpdateChatTopMessage will not be sent, so returned message should be used to update chat top message +-- @bot_user_id Identifier of the bot +-- @chat_id Identifier of the chat +-- @parameter Hidden parameter sent to bot for deep linking (https://api.telegram.org/bots#deep-linking) +-- parameter=start|startgroup or custom as defined by bot creator +local function sendBotStartMessage(bot_user_id, chat_id, parameter, dl_cb, cmd) + tdcli_function ({ + ID = "SendBotStartMessage", + bot_user_id_ = bot_user_id, + chat_id_ = chat_id, + parameter_ = parameter + }, dl_cb, cmd) +end + +M.sendBotStartMessage = sendBotStartMessage + +-- Sends result of the inline query as a message. +-- Returns sent message. +-- UpdateChatTopMessage will not be sent, so returned message should be used to update chat top message. +-- Always clears chat draft message +-- @chat_id Chat to send message +-- @reply_to_message_id Identifier of a message to reply to or 0 +-- @disable_notification Pass true, to disable notification about the message, doesn't works in secret chats +-- @from_background Pass true, if the message is sent from background +-- @query_id Identifier of the inline query +-- @result_id Identifier of the inline result +local function sendInlineQueryResultMessage(chat_id, reply_to_message_id, disable_notification, from_background, query_id, result_id, dl_cb, cmd) + tdcli_function ({ + ID = "SendInlineQueryResultMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + query_id_ = query_id, + result_id_ = result_id + }, dl_cb, cmd) +end + +M.sendInlineQueryResultMessage = sendInlineQueryResultMessage + +-- Forwards previously sent messages. +-- Returns forwarded messages in the same order as message identifiers passed in message_ids. +-- If message can't be forwarded, null will be returned instead of the message. +-- UpdateChatTopMessage will not be sent, so returned messages should be used to update chat top message +-- @chat_id Identifier of a chat to forward messages +-- @from_chat_id Identifier of a chat to forward from +-- @message_ids Identifiers of messages to forward +-- @disable_notification Pass true, to disable notification about the message, doesn't works if messages are forwarded to secret chat +-- @from_background Pass true, if the message is sent from background +local function forwardMessages(chat_id, from_chat_id, message_ids, disable_notification, dl_cb, cmd) + tdcli_function ({ + ID = "ForwardMessages", + chat_id_ = chat_id, + from_chat_id_ = from_chat_id, + message_ids_ = message_ids, -- vector + disable_notification_ = disable_notification, + from_background_ = 1 + }, dl_cb, cmd) +end + +M.forwardMessages = forwardMessages + +-- Changes current ttl setting in a secret chat and sends corresponding message +-- @chat_id Chat identifier +-- @ttl New value of ttl in seconds +local function sendChatSetTtlMessage(chat_id, ttl, dl_cb, cmd) + tdcli_function ({ + ID = "SendChatSetTtlMessage", + chat_id_ = chat_id, + ttl_ = ttl + }, dl_cb, cmd) +end + +M.sendChatSetTtlMessage = sendChatSetTtlMessage + +-- Deletes messages. +-- UpdateDeleteMessages will not be sent for messages deleted through that function +-- @chat_id Chat identifier +-- @message_ids Identifiers of messages to delete +local function deleteMessages(chat_id, message_ids, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteMessages", + chat_id_ = chat_id, + message_ids_ = message_ids -- vector + }, dl_cb, cmd) +end + +M.deleteMessages = deleteMessages + +-- Deletes all messages in the chat sent by the specified user. +-- Works only in supergroup channel chats, needs appropriate privileges +-- @chat_id Chat identifier +-- @user_id User identifier +local function deleteMessagesFromUser(chat_id, user_id, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteMessagesFromUser", + chat_id_ = chat_id, + user_id_ = user_id + }, dl_cb, cmd) +end + +M.deleteMessagesFromUser = deleteMessagesFromUser + +-- Edits text of text or game message. +-- Non-bots can edit message in a limited period of time. +-- Returns edited message after edit is complete server side +-- @chat_id Chat the message belongs to +-- @message_id Identifier of the message +-- @reply_markup Bots only. New message reply markup +-- @input_message_content New text content of the message. Should be of type InputMessageText +local function editMessageText(chat_id, message_id, reply_markup, text, disable_web_page_preview, parse_mode, dl_cb, cmd) + local TextParseMode = getParseMode(parse_mode) + + tdcli_function ({ + ID = "EditMessageText", + chat_id_ = chat_id, + message_id_ = message_id, + reply_markup_ = reply_markup, -- reply_markup:ReplyMarkup + input_message_content_ = { + ID = "InputMessageText", + text_ = text, + disable_web_page_preview_ = disable_web_page_preview, + clear_draft_ = 0, + entities_ = {}, + parse_mode_ = TextParseMode, + }, + }, dl_cb, cmd) +end + +M.editMessageText = editMessageText + +-- Edits message content caption. +-- Non-bots can edit message in a limited period of time. +-- Returns edited message after edit is complete server side +-- @chat_id Chat the message belongs to +-- @message_id Identifier of the message +-- @reply_markup Bots only. New message reply markup +-- @caption New message content caption, 0-200 characters +local function editMessageCaption(chat_id, message_id, reply_markup, caption, dl_cb, cmd) + tdcli_function ({ + ID = "EditMessageCaption", + chat_id_ = chat_id, + message_id_ = message_id, + reply_markup_ = reply_markup, -- reply_markup:ReplyMarkup + caption_ = caption + }, dl_cb, cmd) +end + +M.editMessageCaption = editMessageCaption + +-- Bots only. +-- Edits message reply markup. +-- Returns edited message after edit is complete server side +-- @chat_id Chat the message belongs to +-- @message_id Identifier of the message +-- @reply_markup New message reply markup +local function editMessageReplyMarkup(inline_message_id, reply_markup, caption, dl_cb, cmd) + tdcli_function ({ + ID = "EditInlineMessageCaption", + inline_message_id_ = inline_message_id, + reply_markup_ = reply_markup, -- reply_markup:ReplyMarkup + caption_ = caption + }, dl_cb, cmd) +end + +M.editMessageReplyMarkup = editMessageReplyMarkup + +-- Bots only. +-- Edits text of an inline text or game message sent via bot +-- @inline_message_id Inline message identifier +-- @reply_markup New message reply markup +-- @input_message_content New text content of the message. Should be of type InputMessageText +local function editInlineMessageText(inline_message_id, reply_markup, text, disable_web_page_preview, dl_cb, cmd) + tdcli_function ({ + ID = "EditInlineMessageText", + inline_message_id_ = inline_message_id, + reply_markup_ = reply_markup, -- reply_markup:ReplyMarkup + input_message_content_ = { + ID = "InputMessageText", + text_ = text, + disable_web_page_preview_ = disable_web_page_preview, + clear_draft_ = 0, + entities_ = {} + }, + }, dl_cb, cmd) +end + +M.editInlineMessageText = editInlineMessageText + +-- Bots only. +-- Edits caption of an inline message content sent via bot +-- @inline_message_id Inline message identifier +-- @reply_markup New message reply markup +-- @caption New message content caption, 0-200 characters +local function editInlineMessageCaption(inline_message_id, reply_markup, caption, dl_cb, cmd) + tdcli_function ({ + ID = "EditInlineMessageCaption", + inline_message_id_ = inline_message_id, + reply_markup_ = reply_markup, -- reply_markup:ReplyMarkup + caption_ = caption + }, dl_cb, cmd) +end + +M.editInlineMessageCaption = editInlineMessageCaption + +-- Bots only. +-- Edits reply markup of an inline message sent via bot +-- @inline_message_id Inline message identifier +-- @reply_markup New message reply markup +local function editInlineMessageReplyMarkup(inline_message_id, reply_markup, dl_cb, cmd) + tdcli_function ({ + ID = "EditInlineMessageReplyMarkup", + inline_message_id_ = inline_message_id, + reply_markup_ = reply_markup -- reply_markup:ReplyMarkup + }, dl_cb, cmd) +end + +M.editInlineMessageReplyMarkup = editInlineMessageReplyMarkup + + +-- Sends inline query to a bot and returns its results. +-- Unavailable for bots +-- @bot_user_id Identifier of the bot send query to +-- @chat_id Identifier of the chat, where the query is sent +-- @user_location User location, only if needed +-- @query Text of the query +-- @offset Offset of the first entry to return +local function getInlineQueryResults(bot_user_id, chat_id, latitude, longitude, query, offset, dl_cb, cmd) + tdcli_function ({ + ID = "GetInlineQueryResults", + bot_user_id_ = bot_user_id, + chat_id_ = chat_id, + user_location_ = { + ID = "Location", + latitude_ = latitude, + longitude_ = longitude + }, + query_ = query, + offset_ = offset + }, dl_cb, cmd) +end + +M.getInlineQueryResults = getInlineQueryResults + +-- Bots only. +-- Sets result of the inline query +-- @inline_query_id Identifier of the inline query +-- @is_personal Does result of the query can be cached only for specified user +-- @results Results of the query +-- @cache_time Allowed time to cache results of the query in seconds +-- @next_offset Offset for the next inline query, pass empty string if there is no more results +-- @switch_pm_text If non-empty, this text should be shown on the button, which opens private chat with the bot and sends bot start message with parameter switch_pm_parameter +-- @switch_pm_parameter Parameter for the bot start message +local function answerInlineQuery(inline_query_id, is_personal, cache_time, next_offset, switch_pm_text, switch_pm_parameter, dl_cb, cmd) + tdcli_function ({ + ID = "AnswerInlineQuery", + inline_query_id_ = inline_query_id, + is_personal_ = is_personal, + results_ = results, --vector<InputInlineQueryResult>, + cache_time_ = cache_time, + next_offset_ = next_offset, + switch_pm_text_ = switch_pm_text, + switch_pm_parameter_ = switch_pm_parameter + }, dl_cb, cmd) +end + +M.answerInlineQuery = answerInlineQuery + +-- Sends callback query to a bot and returns answer to it. +-- Unavailable for bots +-- @chat_id Identifier of the chat with a message +-- @message_id Identifier of the message, from which the query is originated +-- @payload Query payload +-- @text Text of the answer +-- @show_alert If true, an alert should be shown to the user instead of a toast +-- @url URL to be open +local function getCallbackQueryAnswer(chat_id, message_id, text, show_alert, url, dl_cb, cmd) + tdcli_function ({ + ID = "GetCallbackQueryAnswer", + chat_id_ = chat_id, + message_id_ = message_id, + payload_ = { + ID = "CallbackQueryAnswer", + text_ = text, + show_alert_ = show_alert, + url_ = url + }, + }, dl_cb, cmd) +end + +M.getCallbackQueryAnswer = getCallbackQueryAnswer + +-- Bots only. +-- Sets result of the callback query +-- @callback_query_id Identifier of the callback query +-- @text Text of the answer +-- @show_alert If true, an alert should be shown to the user instead of a toast +-- @url Url to be opened +-- @cache_time Allowed time to cache result of the query in seconds +local function answerCallbackQuery(callback_query_id, text, show_alert, url, cache_time, dl_cb, cmd) + tdcli_function ({ + ID = "AnswerCallbackQuery", + callback_query_id_ = callback_query_id, + text_ = text, + show_alert_ = show_alert, + url_ = url, + cache_time_ = cache_time + }, dl_cb, cmd) +end + +M.answerCallbackQuery = answerCallbackQuery + +-- Bots only. +-- Updates game score of the specified user in the game +-- @chat_id Chat a message with the game belongs to +-- @message_id Identifier of the message +-- @edit_message True, if message should be edited +-- @user_id User identifier +-- @score New score +-- @force Pass True to update the score even if it decreases. If score is 0, user will be deleted from the high scores table +local function setGameScore(chat_id, message_id, edit_message, user_id, score, force, dl_cb, cmd) + tdcli_function ({ + ID = "SetGameScore", + chat_id_ = chat_id, + message_id_ = message_id, + edit_message_ = edit_message, + user_id_ = user_id, + score_ = score, + force_ = force + }, dl_cb, cmd) +end + +M.setGameScore = setGameScore + +-- Bots only. +-- Updates game score of the specified user in the game +-- @inline_message_id Inline message identifier +-- @edit_message True, if message should be edited +-- @user_id User identifier +-- @score New score +-- @force Pass True to update the score even if it decreases. If score is 0, user will be deleted from the high scores table +local function setInlineGameScore(inline_message_id, edit_message, user_id, score, force, dl_cb, cmd) + tdcli_function ({ + ID = "SetInlineGameScore", + inline_message_id_ = inline_message_id, + edit_message_ = edit_message, + user_id_ = user_id, + score_ = score, + force_ = force + }, dl_cb, cmd) +end + +M.setInlineGameScore = setInlineGameScore + +-- Bots only. +-- Returns game high scores and some part of the score table around of the specified user in the game +-- @chat_id Chat a message with the game belongs to +-- @message_id Identifier of the message +-- @user_id User identifie +local function getGameHighScores(chat_id, message_id, user_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetGameHighScores", + chat_id_ = chat_id, + message_id_ = message_id, + user_id_ = user_id + }, dl_cb, cmd) +end + +M.getGameHighScores = getGameHighScores + +-- Bots only. +-- Returns game high scores and some part of the score table around of the specified user in the game +-- @inline_message_id Inline message identifier +-- @user_id User identifier +local function getInlineGameHighScores(inline_message_id, user_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetInlineGameHighScores", + inline_message_id_ = inline_message_id, + user_id_ = user_id + }, dl_cb, cmd) +end + +M.getInlineGameHighScores = getInlineGameHighScores + +-- Deletes default reply markup from chat. +-- This method needs to be called after one-time keyboard or ForceReply reply markup has been used. +-- UpdateChatReplyMarkup will be send if reply markup will be changed +-- @chat_id Chat identifier +-- @message_id Message identifier of used keyboard +local function deleteChatReplyMarkup(chat_id, message_id, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteChatReplyMarkup", + chat_id_ = chat_id, + message_id_ = message_id + }, dl_cb, cmd) +end + +M.deleteChatReplyMarkup = deleteChatReplyMarkup + +-- Sends notification about user activity in a chat +-- @chat_id Chat identifier +-- @action Action description +-- action = Typing|Cancel|RecordVideo|UploadVideo|RecordVoice|UploadVoice|UploadPhoto|UploadDocument|GeoLocation|ChooseContact|StartPlayGame +local function sendChatAction(chat_id, action, progress, dl_cb, cmd) + tdcli_function ({ + ID = "SendChatAction", + chat_id_ = chat_id, + action_ = { + ID = "SendMessage" .. action .. "Action", + progress_ = progress or 100 + } + }, dl_cb, cmd) +end + +M.sendChatAction = sendChatAction + +-- Sends notification about screenshot taken in a chat. +-- Works only in secret chats +-- @chat_id Chat identifier +local function sendChatScreenshotTakenNotification(chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "SendChatScreenshotTakenNotification", + chat_id_ = chat_id + }, dl_cb, cmd) +end + +M.sendChatScreenshotTakenNotification = sendChatScreenshotTakenNotification + +-- Chat is opened by the user. +-- Many useful activities depends on chat being opened or closed. For example, in channels all updates are received only for opened chats +-- @chat_id Chat identifier +local function openChat(chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "OpenChat", + chat_id_ = chat_id + }, dl_cb, cmd) +end + +M.openChat = openChat + +-- Chat is closed by the user. +-- Many useful activities depends on chat being opened or closed. +-- @chat_id Chat identifier +local function closeChat(chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "CloseChat", + chat_id_ = chat_id + }, dl_cb, cmd) +end + +M.closeChat = closeChat + +-- Messages are viewed by the user. +-- Many useful activities depends on message being viewed. For example, marking messages as read, incrementing of view counter, updating of view counter, removing of deleted messages in channels +-- @chat_id Chat identifier +-- @message_ids Identifiers of viewed messages +local function viewMessages(chat_id, message_ids, dl_cb, cmd) + tdcli_function ({ + ID = "ViewMessages", + chat_id_ = chat_id, + message_ids_ = message_ids -- vector + }, dl_cb, cmd) +end + +M.viewMessages = viewMessages + +-- Message content is opened, for example the user has opened a photo, a video, a document, a location or a venue or have listened to an audio or a voice message +-- @chat_id Chat identifier of the message +-- @message_id Identifier of the message with opened content +local function openMessageContent(chat_id, message_id, dl_cb, cmd) + tdcli_function ({ + ID = "OpenMessageContent", + chat_id_ = chat_id, + message_id_ = message_id + }, dl_cb, cmd) +end + +M.openMessageContent = openMessageContent + +-- Returns existing chat corresponding to the given user +-- @user_id User identifier +local function createPrivateChat(user_id, dl_cb, cmd) + tdcli_function ({ + ID = "CreatePrivateChat", + user_id_ = user_id + }, dl_cb, cmd) +end + +M.createPrivateChat = createPrivateChat + +-- Returns existing chat corresponding to the known group +-- @group_id Group identifier +local function createGroupChat(group_id, dl_cb, cmd) + tdcli_function ({ + ID = "CreateGroupChat", + group_id_ = getChatId(group_id).ID + }, dl_cb, cmd) +end + +M.createGroupChat = createGroupChat + +-- Returns existing chat corresponding to the known channel +-- @channel_id Channel identifier +local function createChannelChat(channel_id, dl_cb, cmd) + tdcli_function ({ + ID = "CreateChannelChat", + channel_id_ = getChatId(channel_id).ID + }, dl_cb, cmd) +end + +M.createChannelChat = createChannelChat + +-- Returns existing chat corresponding to the known secret chat +-- @secret_chat_id SecretChat identifier +local function createSecretChat(secret_chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "CreateSecretChat", + secret_chat_id_ = secret_chat_id + }, dl_cb, cmd) +end + +M.createSecretChat = createSecretChat + +-- Creates new group chat and send corresponding messageGroupChatCreate, returns created chat +-- @user_ids Identifiers of users to add to the group +-- @title Title of new group chat, 0-255 characters +local function createNewGroupChat(user_ids, title, dl_cb, cmd) + tdcli_function ({ + ID = "CreateNewGroupChat", + user_ids_ = user_ids, -- vector + title_ = title + }, dl_cb, cmd) +end + +M.createNewGroupChat = createNewGroupChat + +-- Creates new channel chat and send corresponding messageChannelChatCreate, returns created chat +-- @title Title of new channel chat, 0-255 characters +-- @is_supergroup True, if supergroup chat should be created +-- @about Information about the channel, 0-255 characters +local function createNewChannelChat(title, is_supergroup, about, dl_cb, cmd) + tdcli_function ({ + ID = "CreateNewChannelChat", + title_ = title, + is_supergroup_ = is_supergroup, + about_ = about + }, dl_cb, cmd) +end + +M.createNewChannelChat = createNewChannelChat + +-- Creates new secret chat, returns created chat +-- @user_id Identifier of a user to create secret chat with +local function createNewSecretChat(user_id, dl_cb, cmd) + tdcli_function ({ + ID = "CreateNewSecretChat", + user_id_ = user_id + }, dl_cb, cmd) +end + +M.createNewSecretChat = createNewSecretChat + +-- Creates new channel supergroup chat from existing group chat and send corresponding messageChatMigrateTo and messageChatMigrateFrom. Deactivates group +-- @chat_id Group chat identifier +local function migrateGroupChatToChannelChat(chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "MigrateGroupChatToChannelChat", + chat_id_ = chat_id + }, dl_cb, cmd) +end + +M.migrateGroupChatToChannelChat = migrateGroupChatToChannelChat + +-- Changes chat title. +-- Title can't be changed for private chats. +-- Title will not change until change will be synchronized with the server. +-- Title will not be changed if application is killed before it can send request to the server. +-- There will be update about change of the title on success. Otherwise error will be returned +-- @chat_id Chat identifier +-- @title New title of a chat, 0-255 characters +local function changeChatTitle(chat_id, title, dl_cb, cmd) + tdcli_function ({ + ID = "ChangeChatTitle", + chat_id_ = chat_id, + title_ = title + }, dl_cb, cmd) +end + +M.changeChatTitle = changeChatTitle + +-- Changes chat photo. +-- Photo can't be changed for private chats. +-- Photo will not change until change will be synchronized with the server. +-- Photo will not be changed if application is killed before it can send request to the server. +-- There will be update about change of the photo on success. Otherwise error will be returned +-- @chat_id Chat identifier +-- @photo New chat photo. You can use zero InputFileId to delete photo. Files accessible only by HTTP URL are not acceptable +local function changeChatPhoto(chat_id, photo, dl_cb, cmd) + tdcli_function ({ + ID = "ChangeChatPhoto", + chat_id_ = chat_id, + photo_ = getInputFile(photo) + }, dl_cb, cmd) +end + +M.changeChatPhoto = changeChatPhoto + +-- Changes chat draft message +-- @chat_id Chat identifier +-- @draft_message New draft message, nullable +local function changeChatDraftMessage(chat_id, reply_to_message_id, text, disable_web_page_preview, clear_draft, parse_mode, dl_cb, cmd) + local TextParseMode = getParseMode(parse_mode) + + tdcli_function ({ + ID = "ChangeChatDraftMessage", + chat_id_ = chat_id, + draft_message_ = { + ID = "DraftMessage", + reply_to_message_id_ = reply_to_message_id, + input_message_text_ = { + ID = "InputMessageText", + text_ = text, + disable_web_page_preview_ = disable_web_page_preview, + clear_draft_ = clear_draft, + entities_ = {}, + parse_mode_ = TextParseMode, + }, + }, + }, dl_cb, cmd) +end + +M.changeChatDraftMessage = changeChatDraftMessage + +-- Adds new member to chat. +-- Members can't be added to private or secret chats. +-- Member will not be added until chat state will be synchronized with the server. +-- Member will not be added if application is killed before it can send request to the server +-- @chat_id Chat identifier +-- @user_id Identifier of the user to add +-- @forward_limit Number of previous messages from chat to forward to new member, ignored for channel chats +local function addChatMember(chat_id, user_id, forward_limit, dl_cb, cmd) + tdcli_function ({ + ID = "AddChatMember", + chat_id_ = chat_id, + user_id_ = user_id, + forward_limit_ = forward_limit or 50 + }, dl_cb, cmd) +end + +M.addChatMember = addChatMember + +-- Adds many new members to the chat. +-- Currently, available only for channels. +-- Can't be used to join the channel. +-- Member will not be added until chat state will be synchronized with the server. +-- Member will not be added if application is killed before it can send request to the server +-- @chat_id Chat identifier +-- @user_ids Identifiers of the users to add +local function addChatMembers(chat_id, user_ids, dl_cb, cmd) + tdcli_function ({ + ID = "AddChatMembers", + chat_id_ = chat_id, + user_ids_ = user_ids -- vector + }, dl_cb, cmd) +end + +M.addChatMembers = addChatMembers + +-- Changes status of the chat member, need appropriate privileges. +-- In channel chats, user will be added to chat members if he is yet not a member and there is less than 200 members in the channel. +-- Status will not be changed until chat state will be synchronized with the server. +-- Status will not be changed if application is killed before it can send request to the server +-- @chat_id Chat identifier +-- @user_id Identifier of the user to edit status, bots can be editors in the channel chats +-- @status New status of the member in the chat +-- status = Creator|Editor|Moderator|Member|Left|Kicked +local function changeChatMemberStatus(chat_id, user_id, status, dl_cb, cmd) + tdcli_function ({ + ID = "ChangeChatMemberStatus", + chat_id_ = chat_id, + user_id_ = user_id, + status_ = { + ID = "ChatMemberStatus" .. status + }, + }, dl_cb, cmd) +end + +M.changeChatMemberStatus = changeChatMemberStatus + +-- Returns information about one participant of the chat +-- @chat_id Chat identifier +-- @user_id User identifier +local function getChatMember(chat_id, user_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetChatMember", + chat_id_ = chat_id, + user_id_ = user_id + }, dl_cb, cmd) +end + +M.getChatMember = getChatMember + +-- Asynchronously downloads file from cloud. +-- Updates updateFileProgress will notify about download progress. +-- Update updateFile will notify about successful download +-- @file_id Identifier of file to download +local function downloadFile(file_id, dl_cb, cmd) + tdcli_function ({ + ID = "DownloadFile", + file_id_ = file_id + }, dl_cb, cmd) +end + +M.downloadFile = downloadFile + +-- Stops file downloading. +-- If file already downloaded do nothing. +-- @file_id Identifier of file to cancel download +local function cancelDownloadFile(file_id, dl_cb, cmd) + tdcli_function ({ + ID = "CancelDownloadFile", + file_id_ = file_id + }, dl_cb, cmd) +end + +M.cancelDownloadFile = cancelDownloadFile + +-- Next part of a file was generated +-- @generation_id Identifier of the generation process +-- @ready Number of bytes already generated. Negative number means that generation has failed and should be terminated +local function setFileGenerationProgress(generation_id, ready, dl_cb, cmd) + tdcli_function ({ + ID = "SetFileGenerationProgress", + generation_id_ = generation_id, + ready_ = ready + }, dl_cb, cmd) +end + +M.setFileGenerationProgress = setFileGenerationProgress + +-- Finishes file generation +-- @generation_id Identifier of the generation process +local function finishFileGeneration(generation_id, dl_cb, cmd) + tdcli_function ({ + ID = "FinishFileGeneration", + generation_id_ = generation_id + }, dl_cb, cmd) +end + +M.finishFileGeneration = finishFileGeneration + +-- Generates new chat invite link, previously generated link is revoked. +-- Available for group and channel chats. +-- Only creator of the chat can export chat invite link +-- @chat_id Chat identifier +local function exportChatInviteLink(chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "ExportChatInviteLink", + chat_id_ = chat_id + }, dl_cb, cmd) +end + +M.exportChatInviteLink = exportChatInviteLink + +-- Checks chat invite link for validness and returns information about the corresponding chat +-- @invite_link Invite link to check. Should begin with "https://telegram.me/joinchat/" +local function checkChatInviteLink(link, dl_cb, cmd) + tdcli_function ({ + ID = "CheckChatInviteLink", + invite_link_ = link + }, dl_cb, cmd) +end + +M.checkChatInviteLink = checkChatInviteLink + +-- Imports chat invite link, adds current user to a chat if possible. +-- Member will not be added until chat state will be synchronized with the server. +-- Member will not be added if application is killed before it can send request to the server +-- @invite_link Invite link to import. Should begin with "https://telegram.me/joinchat/" +local function importChatInviteLink(invite_link, dl_cb, cmd) + tdcli_function ({ + ID = "ImportChatInviteLink", + invite_link_ = invite_link + }, dl_cb, cmd) +end + +M.importChatInviteLink = importChatInviteLink + +-- Adds user to black list +-- @user_id User identifier +local function blockUser(user_id, dl_cb, cmd) + tdcli_function ({ + ID = "BlockUser", + user_id_ = user_id + }, dl_cb, cmd) +end + +M.blockUser = blockUser + +-- Removes user from black list +-- @user_id User identifier +local function unblockUser(user_id, dl_cb, cmd) + tdcli_function ({ + ID = "UnblockUser", + user_id_ = user_id + }, dl_cb, cmd) +end + +M.unblockUser = unblockUser + +-- Returns users blocked by the current user +-- @offset Number of users to skip in result, must be non-negative +-- @limit Maximum number of users to return, can't be greater than 100 +local function getBlockedUsers(offset, limit, dl_cb, cmd) + tdcli_function ({ + ID = "GetBlockedUsers", + offset_ = offset, + limit_ = limit + }, dl_cb, cmd) +end + +M.getBlockedUsers = getBlockedUsers + +-- Adds new contacts/edits existing contacts, contacts user identifiers are ignored. +-- Returns list of corresponding users in the same order as input contacts. +-- If contact doesn't registered in Telegram, user with id == 0 will be returned +-- @contacts List of contacts to import/edit +local function importContacts(phone_number, first_name, last_name, user_id, dl_cb, cmd) + tdcli_function ({ + ID = "ImportContacts", + contacts_ = {[0] = { + phone_number_ = tostring(phone_number), + first_name_ = tostring(first_name), + last_name_ = tostring(last_name), + user_id_ = user_id + }, + }, + }, dl_cb, cmd) +end + +M.importContacts = importContacts + +-- Searches for specified query in the first name, last name and username of the known user contacts +-- @query Query to search for, can be empty to return all contacts +-- @limit Maximum number of users to be returned +local function searchContacts(query, limit, dl_cb, cmd) + tdcli_function ({ + ID = "SearchContacts", + query_ = query, + limit_ = limit + }, dl_cb, cmd) +end + +M.searchContacts = searchContacts + +-- Deletes users from contacts list +-- @user_ids Identifiers of users to be deleted +local function deleteContacts(user_ids, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteContacts", + user_ids_ = user_ids -- vector + }, dl_cb, cmd) +end + +M.deleteContacts = deleteContacts + +-- Returns profile photos of the user. +-- Result of this query can't be invalidated, so it must be used with care +-- @user_id User identifier +-- @offset Photos to skip, must be non-negative +-- @limit Maximum number of photos to be returned, can't be greater than 100 +local function getUserProfilePhotos(user_id, offset, limit, dl_cb, cmd) + tdcli_function ({ + ID = "GetUserProfilePhotos", + user_id_ = user_id, + offset_ = offset, + limit_ = limit + }, dl_cb, cmd) +end + +M.getUserProfilePhotos = getUserProfilePhotos + +-- Returns stickers corresponding to given emoji +-- @emoji String representation of emoji. If empty, returns all known stickers +local function getStickers(emoji, dl_cb, cmd) + tdcli_function ({ + ID = "GetStickers", + emoji_ = emoji + }, dl_cb, cmd) +end + +M.getStickers = getStickers + +-- Returns list of installed sticker sets without archived sticker sets +-- @is_masks Pass true to return masks, pass false to return stickers +local function getStickerSets(is_masks, dl_cb, cmd) + tdcli_function ({ + ID = "GetStickerSets", + is_masks_ = is_masks + }, dl_cb, cmd) +end + +M.getStickerSets = getStickerSets + +-- Returns list of archived sticker sets +-- @is_masks Pass true to return masks, pass false to return stickers +-- @offset_sticker_set_id Identifier of the sticker set from which return the result +-- @limit Maximum number of sticker sets to return +local function getArchivedStickerSets(is_masks, offset_sticker_set_id, limit, dl_cb, cmd) + tdcli_function ({ + ID = "GetArchivedStickerSets", + is_masks_ = is_masks, + offset_sticker_set_id_ = offset_sticker_set_id, + limit_ = limit + }, dl_cb, cmd) +end + +M.getArchivedStickerSets = getArchivedStickerSets + +-- Returns list of trending sticker sets +local function getTrendingStickerSets(dl_cb, cmd) + tdcli_function ({ + ID = "GetTrendingStickerSets" + }, dl_cb, cmd) +end + +M.getTrendingStickerSets = getTrendingStickerSets + +-- Returns list of sticker sets attached to a file, currently only photos and videos can have attached sticker sets +-- @file_id File identifier +local function getAttachedStickerSets(file_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetAttachedStickerSets", + file_id_ = file_id + }, dl_cb, cmd) +end + +M.getAttachedStickerSets = getAttachedStickerSets + +-- Returns information about sticker set by its identifier +-- @set_id Identifier of the sticker set +local function getStickerSet(set_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetStickerSet", + set_id_ = set_id + }, dl_cb, cmd) +end + +M.getStickerSet = getStickerSet + +-- Searches sticker set by its short name +-- @name Name of the sticker set +local function searchStickerSet(name, dl_cb, cmd) + tdcli_function ({ + ID = "SearchStickerSet", + name_ = name + }, dl_cb, cmd) +end + +M.searchStickerSet = searchStickerSet + +-- Installs/uninstalls or enables/archives sticker set. +-- Official sticker set can't be uninstalled, but it can be archived +-- @set_id Identifier of the sticker set +-- @is_installed New value of is_installed +-- @is_archived New value of is_archived +local function updateStickerSet(set_id, is_installed, is_archived, dl_cb, cmd) + tdcli_function ({ + ID = "UpdateStickerSet", + set_id_ = set_id, + is_installed_ = is_installed, + is_archived_ = is_archived + }, dl_cb, cmd) +end + +M.updateStickerSet = updateStickerSet + +-- Trending sticker sets are viewed by the user +-- @sticker_set_ids Identifiers of viewed trending sticker sets +local function viewTrendingStickerSets(sticker_set_ids, dl_cb, cmd) + tdcli_function ({ + ID = "ViewTrendingStickerSets", + sticker_set_ids_ = sticker_set_ids -- vector + }, dl_cb, cmd) +end + +M.viewTrendingStickerSets = viewTrendingStickerSets + +-- Changes the order of installed sticker sets +-- @is_masks Pass true to change masks order, pass false to change stickers order +-- @sticker_set_ids Identifiers of installed sticker sets in the new right order +local function reorderStickerSets(is_masks, sticker_set_ids, dl_cb, cmd) + tdcli_function ({ + ID = "ReorderStickerSets", + is_masks_ = is_masks, + sticker_set_ids_ = sticker_set_ids -- vector + }, dl_cb, cmd) +end + +M.reorderStickerSets = reorderStickerSets + +-- Returns list of recently used stickers +-- @is_attached Pass true to return stickers and masks recently attached to photo or video files, pass false to return recently sent stickers +local function getRecentStickers(is_attached, dl_cb, cmd) + tdcli_function ({ + ID = "GetRecentStickers", + is_attached_ = is_attached + }, dl_cb, cmd) +end + +M.getRecentStickers = getRecentStickers + +-- Manually adds new sticker to the list of recently used stickers. +-- New sticker is added to the beginning of the list. +-- If the sticker is already in the list, at first it is removed from the list +-- @is_attached Pass true to add the sticker to the list of stickers recently attached to photo or video files, pass false to add the sticker to the list of recently sent stickers +-- @sticker Sticker file to add +local function addRecentSticker(is_attached, sticker, dl_cb, cmd) + tdcli_function ({ + ID = "AddRecentSticker", + is_attached_ = is_attached, + sticker_ = getInputFile(sticker) + }, dl_cb, cmd) +end + +M.addRecentSticker = addRecentSticker + +-- Removes a sticker from the list of recently used stickers +-- @is_attached Pass true to remove the sticker from the list of stickers recently attached to photo or video files, pass false to remove the sticker from the list of recently sent stickers +-- @sticker Sticker file to delete +local function deleteRecentSticker(is_attached, sticker, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteRecentSticker", + is_attached_ = is_attached, + sticker_ = getInputFile(sticker) + }, dl_cb, cmd) +end + +M.deleteRecentSticker = deleteRecentSticker + +-- Clears list of recently used stickers +-- @is_attached Pass true to clear list of stickers recently attached to photo or video files, pass false to clear the list of recently sent stickers +local function clearRecentStickers(is_attached, dl_cb, cmd) + tdcli_function ({ + ID = "ClearRecentStickers", + is_attached_ = is_attached + }, dl_cb, cmd) +end + +M.clearRecentStickers = clearRecentStickers + +-- Returns emojis corresponding to a sticker +-- @sticker Sticker file identifier +local function getStickerEmojis(sticker, dl_cb, cmd) + tdcli_function ({ + ID = "GetStickerEmojis", + sticker_ = getInputFile(sticker) + }, dl_cb, cmd) +end + +M.getStickerEmojis = getStickerEmojis + +-- Returns saved animations +local function getSavedAnimations(dl_cb, cmd) + tdcli_function ({ + ID = "GetSavedAnimations", + }, dl_cb, cmd) +end + +M.getSavedAnimations = getSavedAnimations + +-- Manually adds new animation to the list of saved animations. +-- New animation is added to the beginning of the list. +-- If the animation is already in the list, at first it is removed from the list. +-- Only non-secret video animations with MIME type "video/mp4" can be added to the list +-- @animation Animation file to add. Only known to server animations (i. e. successfully sent via message) can be added to the list +local function addSavedAnimation(animation, dl_cb, cmd) + tdcli_function ({ + ID = "AddSavedAnimation", + animation_ = getInputFile(animation) + }, dl_cb, cmd) +end + +M.addSavedAnimation = addSavedAnimation + +-- Removes animation from the list of saved animations +-- @animation Animation file to delete +local function deleteSavedAnimation(animation, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteSavedAnimation", + animation_ = getInputFile(animation) + }, dl_cb, cmd) +end + +M.deleteSavedAnimation = deleteSavedAnimation + +-- Returns up to 20 recently used inline bots in the order of the last usage +local function getRecentInlineBots(dl_cb, cmd) + tdcli_function ({ + ID = "GetRecentInlineBots", + }, dl_cb, cmd) +end + +M.getRecentInlineBots = getRecentInlineBots + +-- Get web page preview by text of the message. +-- Do not call this function to often +-- @message_text Message text +local function getWebPagePreview(message_text, dl_cb, cmd) + tdcli_function ({ + ID = "GetWebPagePreview", + message_text_ = message_text + }, dl_cb, cmd) +end + +M.getWebPagePreview = getWebPagePreview + +-- Returns notification settings for a given scope +-- @scope Scope to return information about notification settings +-- scope = Chat(chat_id)|PrivateChats|GroupChats|AllChats| +local function getNotificationSettings(scope, chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetNotificationSettings", + scope_ = { + ID = 'NotificationSettingsFor' .. scope, + chat_id_ = chat_id or nil + }, + }, dl_cb, cmd) +end + +M.getNotificationSettings = getNotificationSettings + +-- Changes notification settings for a given scope +-- @scope Scope to change notification settings +-- @notification_settings New notification settings for given scope +-- scope = Chat(chat_id)|PrivateChats|GroupChats|AllChats| +local function setNotificationSettings(scope, chat_id, mute_for, show_preview, dl_cb, cmd) + tdcli_function ({ + ID = "SetNotificationSettings", + scope_ = { + ID = 'NotificationSettingsFor' .. scope, + chat_id_ = chat_id or nil + }, + notification_settings_ = { + ID = "NotificationSettings", + mute_for_ = mute_for, + sound_ = "default", + show_preview_ = show_preview + } + }, dl_cb, cmd) +end + +M.setNotificationSettings = setNotificationSettings + +-- Resets all notification settings to the default value. +-- By default the only muted chats are supergroups, sound is set to 'default' and message previews are showed +local function resetAllNotificationSettings(dl_cb, cmd) + tdcli_function ({ + ID = "ResetAllNotificationSettings" + }, dl_cb, cmd) +end + +M.resetAllNotificationSettings = resetAllNotificationSettings + +-- Uploads new profile photo for logged in user. +-- Photo will not change until change will be synchronized with the server. +-- Photo will not be changed if application is killed before it can send request to the server. +-- If something changes, updateUser will be sent +-- @photo_path Path to new profile photo +local function setProfilePhoto(photo_path, dl_cb, cmd) + tdcli_function ({ + ID = "SetProfilePhoto", + photo_path_ = photo_path + }, dl_cb, cmd) +end + +M.setProfilePhoto = setProfilePhoto + +-- Deletes profile photo. +-- If something changes, updateUser will be sent +-- @profile_photo_id Identifier of profile photo to delete +local function deleteProfilePhoto(profile_photo_id, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteProfilePhoto", + profile_photo_id_ = profile_photo_id + }, dl_cb, cmd) +end + +M.deleteProfilePhoto = deleteProfilePhoto + +-- Changes first and last names of logged in user. +-- If something changes, updateUser will be sent +-- @first_name New value of user first name, 1-255 characters +-- @last_name New value of optional user last name, 0-255 characters +local function changeName(first_name, last_name, dl_cb, cmd) + tdcli_function ({ + ID = "ChangeName", + first_name_ = first_name, + last_name_ = last_name + }, dl_cb, cmd) +end + +M.changeName = changeName + +-- Changes about information of logged in user +-- @about New value of userFull.about, 0-255 characters +local function changeAbout(about, dl_cb, cmd) + tdcli_function ({ + ID = "ChangeAbout", + about_ = about + }, dl_cb, cmd) +end + +M.changeAbout = changeAbout + +-- Changes username of logged in user. +-- If something changes, updateUser will be sent +-- @username New value of username. Use empty string to remove username +local function changeUsername(username, dl_cb, cmd) + tdcli_function ({ + ID = "ChangeUsername", + username_ = username + }, dl_cb, cmd) +end + +M.changeUsername = changeUsername + +-- Changes user's phone number and sends authentication code to the new user's phone number. +-- Returns authStateWaitCode with information about sent code on success +-- @phone_number New user's phone number in any reasonable format +-- @allow_flash_call Pass True, if code can be sent via flash call to the specified phone number +-- @is_current_phone_number Pass true, if the phone number is used on the current device. Ignored if allow_flash_call is False +local function changePhoneNumber(phone_number, allow_flash_call, is_current_phone_number, dl_cb, cmd) + tdcli_function ({ + ID = "ChangePhoneNumber", + phone_number_ = phone_number, + allow_flash_call_ = allow_flash_call, + is_current_phone_number_ = is_current_phone_number + }, dl_cb, cmd) +end + +M.changePhoneNumber = changePhoneNumber + +-- Resends authentication code sent to change user's phone number. +-- Works only if in previously received authStateWaitCode next_code_type was not null. +-- Returns authStateWaitCode on success +local function resendChangePhoneNumberCode(dl_cb, cmd) + tdcli_function ({ + ID = "ResendChangePhoneNumberCode", + }, dl_cb, cmd) +end + +M.resendChangePhoneNumberCode = resendChangePhoneNumberCode + +-- Checks authentication code sent to change user's phone number. +-- Returns authStateOk on success +-- @code Verification code from SMS, voice call or flash call +local function checkChangePhoneNumberCode(code, dl_cb, cmd) + tdcli_function ({ + ID = "CheckChangePhoneNumberCode", + code_ = code + }, dl_cb, cmd) +end + +M.checkChangePhoneNumberCode = checkChangePhoneNumberCode + +-- Returns all active sessions of logged in user +local function getActiveSessions(dl_cb, cmd) + tdcli_function ({ + ID = "GetActiveSessions", + }, dl_cb, cmd) +end + +M.getActiveSessions = getActiveSessions + +-- Terminates another session of logged in user +-- @session_id Session identifier +local function terminateSession(session_id, dl_cb, cmd) + tdcli_function ({ + ID = "TerminateSession", + session_id_ = session_id + }, dl_cb, cmd) +end + +M.terminateSession = terminateSession + +-- Terminates all other sessions of logged in user +local function terminateAllOtherSessions(dl_cb, cmd) + tdcli_function ({ + ID = "TerminateAllOtherSessions", + }, dl_cb, cmd) +end + +M.terminateAllOtherSessions = terminateAllOtherSessions + +-- Gives or revokes all members of the group editor rights. +-- Needs creator privileges in the group +-- @group_id Identifier of the group +-- @anyone_can_edit New value of anyone_can_edit +local function toggleGroupEditors(group_id, anyone_can_edit, dl_cb, cmd) + tdcli_function ({ + ID = "ToggleGroupEditors", + group_id_ = getChatId(group_id).ID, + anyone_can_edit_ = anyone_can_edit + }, dl_cb, cmd) +end + +M.toggleGroupEditors = toggleGroupEditors + +-- Changes username of the channel. +-- Needs creator privileges in the channel +-- @channel_id Identifier of the channel +-- @username New value of username. Use empty string to remove username +local function changeChannelUsername(channel_id, username, dl_cb, cmd) + tdcli_function ({ + ID = "ChangeChannelUsername", + channel_id_ = getChatId(channel_id).ID, + username_ = username + }, dl_cb, cmd) +end + +M.changeChannelUsername = changeChannelUsername + +-- Gives or revokes right to invite new members to all current members of the channel. +-- Needs creator privileges in the channel. +-- Available only for supergroups +-- @channel_id Identifier of the channel +-- @anyone_can_invite New value of anyone_can_invite +local function toggleChannelInvites(channel_id, anyone_can_invite, dl_cb, cmd) + tdcli_function ({ + ID = "ToggleChannelInvites", + channel_id_ = getChatId(channel_id).ID, + anyone_can_invite_ = anyone_can_invite + }, dl_cb, cmd) +end + +M.toggleChannelInvites = toggleChannelInvites + +-- Enables or disables sender signature on sent messages in the channel. +-- Needs creator privileges in the channel. +-- Not available for supergroups +-- @channel_id Identifier of the channel +-- @sign_messages New value of sign_messages +local function toggleChannelSignMessages(channel_id, sign_messages, dl_cb, cmd) + tdcli_function ({ + ID = "ToggleChannelSignMessages", + channel_id_ = getChatId(channel_id).ID, + sign_messages_ = sign_messages + }, dl_cb, cmd) +end + +M.toggleChannelSignMessages = toggleChannelSignMessages + +-- Changes information about the channel. +-- Needs creator privileges in the broadcast channel or editor privileges in the supergroup channel +-- @channel_id Identifier of the channel +-- @about New value of about, 0-255 characters +local function changeChannelAbout(channel_id, about, dl_cb, cmd) + tdcli_function ({ + ID = "ChangeChannelAbout", + channel_id_ = getChatId(channel_id).ID, + about_ = about + }, dl_cb, cmd) +end + +M.changeChannelAbout = changeChannelAbout + +-- Pins a message in a supergroup channel chat. +-- Needs editor privileges in the channel +-- @channel_id Identifier of the channel +-- @message_id Identifier of the new pinned message +-- @disable_notification True, if there should be no notification about the pinned message +local function pinChannelMessage(channel_id, message_id, disable_notification, dl_cb, cmd) + tdcli_function ({ + ID = "PinChannelMessage", + channel_id_ = getChatId(channel_id).ID, + message_id_ = message_id, + disable_notification_ = disable_notification + }, dl_cb, cmd) +end + +M.pinChannelMessage = pinChannelMessage + +-- Removes pinned message in the supergroup channel. +-- Needs editor privileges in the channel +-- @channel_id Identifier of the channel +local function unpinChannelMessage(channel_id, dl_cb, cmd) + tdcli_function ({ + ID = "UnpinChannelMessage", + channel_id_ = getChatId(channel_id).ID + }, dl_cb, cmd) +end + +M.unpinChannelMessage = unpinChannelMessage + +-- Reports some supergroup channel messages from a user as spam messages +-- @channel_id Channel identifier +-- @user_id User identifier +-- @message_ids Identifiers of messages sent in the supergroup by the user, the list should be non-empty +local function reportChannelSpam(channel_id, user_id, message_ids, dl_cb, cmd) + tdcli_function ({ + ID = "ReportChannelSpam", + channel_id_ = getChatId(channel_id).ID, + user_id_ = user_id, + message_ids_ = message_ids -- vector + }, dl_cb, cmd) +end + +M.reportChannelSpam = reportChannelSpam + +-- Returns information about channel members or kicked from channel users. +-- Can be used only if channel_full->can_get_members == true +-- @channel_id Identifier of the channel +-- @filter Kind of channel users to return, defaults to channelMembersRecent +-- @offset Number of channel users to skip +-- @limit Maximum number of users be returned, can't be greater than 200 +-- filter = Recent|Administrators|Kicked|Bots +local function getChannelMembers(channel_id, offset, filter, limit, dl_cb, cmd) + if not limit or limit > 200 then + limit = 200 + end + + tdcli_function ({ + ID = "GetChannelMembers", + channel_id_ = getChatId(channel_id).ID, + filter_ = { + ID = "ChannelMembers" .. filter + }, + offset_ = offset, + limit_ = limit + }, dl_cb, cmd) +end + +M.getChannelMembers = getChannelMembers + +-- Deletes channel along with all messages in corresponding chat. +-- Releases channel username and removes all members. +-- Needs creator privileges in the channel. +-- Channels with more than 1000 members can't be deleted +-- @channel_id Identifier of the channel +local function deleteChannel(channel_id, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteChannel", + channel_id_ = getChatId(channel_id).ID + }, dl_cb, cmd) +end + +M.deleteChannel = deleteChannel + +-- Returns list of created public channels +local function getCreatedPublicChannels(dl_cb, cmd) + tdcli_function ({ + ID = "GetCreatedPublicChannels" + }, dl_cb, cmd) +end + +M.getCreatedPublicChannels = getCreatedPublicChannels + +-- Closes secret chat +-- @secret_chat_id Secret chat identifier +local function closeSecretChat(secret_chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "CloseSecretChat", + secret_chat_id_ = secret_chat_id + }, dl_cb, cmd) +end + +M.closeSecretChat = closeSecretChat + +-- Returns user that can be contacted to get support +local function getSupportUser(dl_cb, cmd) + tdcli_function ({ + ID = "GetSupportUser", + }, dl_cb, cmd) +end + +M.getSupportUser = getSupportUser + +-- Returns background wallpapers +local function getWallpapers(dl_cb, cmd) + tdcli_function ({ + ID = "GetWallpapers", + }, dl_cb, cmd) +end + +M.getWallpapers = getWallpapers + +-- Registers current used device for receiving push notifications +-- @device_token Device token +-- device_token = apns|gcm|mpns|simplePush|ubuntuPhone|blackberry +local function registerDevice(device_token, token, device_token_set, dl_cb, cmd) + local dToken = {ID = device_token .. 'DeviceToken', token_ = token} + + if device_token_set then + dToken = {ID = "DeviceTokenSet", token_ = device_token_set} -- tokens:vector<DeviceToken> + end + + tdcli_function ({ + ID = "RegisterDevice", + device_token_ = dToken + }, dl_cb, cmd) +end + +M.registerDevice = registerDevice + +-- Returns list of used device tokens +local function getDeviceTokens(dl_cb, cmd) + tdcli_function ({ + ID = "GetDeviceTokens", + }, dl_cb, cmd) +end + +M.getDeviceTokens = getDeviceTokens + +-- Changes privacy settings +-- @key Privacy key +-- @rules New privacy rules +-- @privacyKeyUserStatus Privacy key for managing visibility of the user status +-- @privacyKeyChatInvite Privacy key for managing ability of invitation of the user to chats +-- @privacyRuleAllowAll Rule to allow all users +-- @privacyRuleAllowContacts Rule to allow all user contacts +-- @privacyRuleAllowUsers Rule to allow specified users +-- @user_ids User identifiers +-- @privacyRuleDisallowAll Rule to disallow all users +-- @privacyRuleDisallowContacts Rule to disallow all user contacts +-- @privacyRuleDisallowUsers Rule to disallow all specified users +-- key = UserStatus|ChatInvite +-- rules = AllowAll|AllowContacts|AllowUsers(user_ids)|DisallowAll|DisallowContacts|DisallowUsers(user_ids) +local function setPrivacy(key, rule, allowed_user_ids, disallowed_user_ids, dl_cb, cmd) + local rules = {[0] = {ID = 'PrivacyRule' .. rule}} + + if allowed_user_ids then + rules = { + { + ID = 'PrivacyRule' .. rule + }, + [0] = { + ID = "PrivacyRuleAllowUsers", + user_ids_ = allowed_user_ids -- vector + }, + } + end + if disallowed_user_ids then + rules = { + { + ID = 'PrivacyRule' .. rule + }, + [0] = { + ID = "PrivacyRuleDisallowUsers", + user_ids_ = disallowed_user_ids -- vector + }, + } + end + if allowed_user_ids and disallowed_user_ids then + rules = { + { + ID = 'PrivacyRule' .. rule + }, + { + ID = "PrivacyRuleAllowUsers", + user_ids_ = allowed_user_ids + }, + [0] = { + ID = "PrivacyRuleDisallowUsers", + user_ids_ = disallowed_user_ids + }, + } + end + tdcli_function ({ + ID = "SetPrivacy", + key_ = { + ID = 'PrivacyKey' .. key + }, + rules_ = { + ID = "PrivacyRules", + rules_ = rules + }, + }, dl_cb, cmd) +end + +M.setPrivacy = setPrivacy + +-- Returns current privacy settings +-- @key Privacy key +-- key = UserStatus|ChatInvite +local function getPrivacy(key, dl_cb, cmd) + tdcli_function ({ + ID = "GetPrivacy", + key_ = { + ID = "PrivacyKey" .. key + }, + }, dl_cb, cmd) +end + +M.getPrivacy = getPrivacy + +-- Returns value of an option by its name. +-- See list of available options on https://core.telegram.org/tdlib/options +-- @name Name of the option +local function getOption(name, dl_cb, cmd) + tdcli_function ({ + ID = "GetOption", + name_ = name + }, dl_cb, cmd) +end + +M.getOption = getOption + +-- Sets value of an option. +-- See list of available options on https://core.telegram.org/tdlib/options. +-- Only writable options can be set +-- @name Name of the option +-- @value New value of the option +local function setOption(name, option, value, dl_cb, cmd) + tdcli_function ({ + ID = "SetOption", + name_ = name, + value_ = { + ID = 'Option' .. option, + value_ = value + }, + }, dl_cb, cmd) +end + +M.setOption = setOption + +-- Changes period of inactivity, after which the account of currently logged in user will be automatically deleted +-- @ttl New account TTL +local function changeAccountTtl(days, dl_cb, cmd) + tdcli_function ({ + ID = "ChangeAccountTtl", + ttl_ = { + ID = "AccountTtl", + days_ = days + }, + }, dl_cb, cmd) +end + +M.changeAccountTtl = changeAccountTtl + +-- Returns period of inactivity, after which the account of currently logged in user will be automatically deleted +local function getAccountTtl(dl_cb, cmd) + tdcli_function ({ + ID = "GetAccountTtl", + }, dl_cb, cmd) +end + +M.getAccountTtl = getAccountTtl + +-- Deletes the account of currently logged in user, deleting from the server all information associated with it. +-- Account's phone number can be used to create new account, but only once in two weeks +-- @reason Optional reason of account deletion +local function deleteAccount(reason, dl_cb, cmd) + tdcli_function ({ + ID = "DeleteAccount", + reason_ = reason + }, dl_cb, cmd) +end + +M.deleteAccount = deleteAccount + +-- Returns current chat report spam state +-- @chat_id Chat identifier +local function getChatReportSpamState(chat_id, dl_cb, cmd) + tdcli_function ({ + ID = "GetChatReportSpamState", + chat_id_ = chat_id + }, dl_cb, cmd) +end + +M.getChatReportSpamState = getChatReportSpamState + +-- Reports chat as a spam chat or as not a spam chat. +-- Can be used only if ChatReportSpamState.can_report_spam is true. +-- After this request ChatReportSpamState.can_report_spam became false forever +-- @chat_id Chat identifier +-- @is_spam_chat If true, chat will be reported as a spam chat, otherwise it will be marked as not a spam chat +local function changeChatReportSpamState(chat_id, is_spam_chat, dl_cb, cmd) + tdcli_function ({ + ID = "ChangeChatReportSpamState", + chat_id_ = chat_id, + is_spam_chat_ = is_spam_chat + }, dl_cb, cmd) +end + +M.changeChatReportSpamState = changeChatReportSpamState + +-- Bots only. +-- Informs server about number of pending bot updates if they aren't processed for a long time +-- @pending_update_count Number of pending updates +-- @error_message Last error's message +local function setBotUpdatesStatus(pending_update_count, error_message, dl_cb, cmd) + tdcli_function ({ + ID = "SetBotUpdatesStatus", + pending_update_count_ = pending_update_count, + error_message_ = error_message + }, dl_cb, cmd) +end + +M.setBotUpdatesStatus = setBotUpdatesStatus + +-- Returns Ok after specified amount of the time passed +-- @seconds Number of seconds before that function returns +local function setAlarm(seconds, dl_cb, cmd) + tdcli_function ({ + ID = "SetAlarm", + seconds_ = seconds + }, dl_cb, cmd) +end + +M.setAlarm = setAlarm + +-- Text message +-- @text Text to send +-- @disable_notification Pass true, to disable notification about the message, doesn't works in secret chats +-- @from_background Pass true, if the message is sent from background +-- @reply_markup Bots only. Markup for replying to message +-- @disable_web_page_preview Pass true to disable rich preview for link in the message text +-- @clear_draft Pass true if chat draft message should be deleted +-- @entities Bold, Italic, Code, Pre, PreCode and TextUrl entities contained in the text. Non-bot users can't use TextUrl entities. Can't be used with non-null parse_mode +-- @parse_mode Text parse mode, nullable. Can't be used along with enitities +local function sendMessage(chat_id, reply_to_message_id, disable_notification, text, disable_web_page_preview, parse_mode) + local TextParseMode = getParseMode(parse_mode) + + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = 1, + reply_markup_ = nil, + input_message_content_ = { + ID = "InputMessageText", + text_ = text, + disable_web_page_preview_ = disable_web_page_preview, + clear_draft_ = 0, + entities_ = {}, + parse_mode_ = TextParseMode, + }, + }, dl_cb, nil) +end + +M.sendMessage = sendMessage + +-- Animation message +-- @animation Animation file to send +-- @thumb Animation thumb, if available +-- @width Width of the animation, may be replaced by the server +-- @height Height of the animation, may be replaced by the server +-- @caption Animation caption, 0-200 characters +local function sendAnimation(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, animation, width, height, caption, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageAnimation", + animation_ = getInputFile(animation), + --thumb_ = { + --ID = "InputThumb", + --path_ = path, + --width_ = width, + --height_ = height + --}, + width_ = width or '', + height_ = height or '', + caption_ = caption or '' + }, + }, dl_cb, cmd) +end + +M.sendAnimation = sendAnimation + +-- Audio message +-- @audio Audio file to send +-- @album_cover_thumb Thumb of the album's cover, if available +-- @duration Duration of audio in seconds, may be replaced by the server +-- @title Title of the audio, 0-64 characters, may be replaced by the server +-- @performer Performer of the audio, 0-64 characters, may be replaced by the server +-- @caption Audio caption, 0-200 characters +local function sendAudio(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, audio, duration, title, performer, caption, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageAudio", + audio_ = getInputFile(audio), + --album_cover_thumb_ = { + --ID = "InputThumb", + --path_ = path, + --width_ = width, + --height_ = height + --}, + duration_ = duration or '', + title_ = title or '', + performer_ = performer or '', + caption_ = caption or '' + }, + }, dl_cb, cmd) +end + +M.sendAudio = sendAudio + +-- Document message +-- @document Document to send +-- @thumb Document thumb, if available +-- @caption Document caption, 0-200 characters +local function sendDocument(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, document, caption, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageDocument", + document_ = getInputFile(document), + --thumb_ = { + --ID = "InputThumb", + --path_ = path, + --width_ = width, + --height_ = height + --}, + caption_ = caption + }, + }, dl_cb, cmd) +end + +M.sendDocument = sendDocument + +-- Photo message +-- @photo Photo to send +-- @caption Photo caption, 0-200 characters +local function sendPhoto(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, photo, caption, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessagePhoto", + photo_ = getInputFile(photo), + added_sticker_file_ids_ = {}, + width_ = 0, + height_ = 0, + caption_ = caption + }, + }, dl_cb, cmd) +end + +M.sendPhoto = sendPhoto + +-- Sticker message +-- @sticker Sticker to send +-- @thumb Sticker thumb, if available +local function sendSticker(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, sticker, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageSticker", + sticker_ = getInputFile(sticker), + --thumb_ = { + --ID = "InputThumb", + --path_ = path, + --width_ = width, + --height_ = height + --}, + }, + }, dl_cb, cmd) +end + +M.sendSticker = sendSticker + +-- Video message +-- @video Video to send +-- @thumb Video thumb, if available +-- @duration Duration of video in seconds +-- @width Video width +-- @height Video height +-- @caption Video caption, 0-200 characters +local function sendVideo(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, video, duration, width, height, caption, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageVideo", + video_ = getInputFile(video), + --thumb_ = { + --ID = "InputThumb", + --path_ = path, + --width_ = width, + --height_ = height + --}, + added_sticker_file_ids_ = {}, + duration_ = duration or '', + width_ = width or '', + height_ = height or '', + caption_ = caption or '' + }, + }, dl_cb, cmd) +end + +M.sendVideo = sendVideo + +-- Voice message +-- @voice Voice file to send +-- @duration Duration of voice in seconds +-- @waveform Waveform representation of the voice in 5-bit format +-- @caption Voice caption, 0-200 characters +local function sendVoice(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, voice, duration, waveform, caption, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageVoice", + voice_ = getInputFile(voice), + duration_ = duration or '', + waveform_ = waveform or '', + caption_ = caption or '' + }, + }, dl_cb, cmd) +end + +M.sendVoice = sendVoice + +-- Message with location +-- @latitude Latitude of location in degrees as defined by sender +-- @longitude Longitude of location in degrees as defined by sender +local function sendLocation(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, latitude, longitude, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageLocation", + location_ = { + ID = "Location", + latitude_ = latitude, + longitude_ = longitude + }, + }, + }, dl_cb, cmd) +end + +M.sendLocation = sendLocation + +-- Message with information about venue +-- @venue Venue to send +-- @latitude Latitude of location in degrees as defined by sender +-- @longitude Longitude of location in degrees as defined by sender +-- @title Venue name as defined by sender +-- @address Venue address as defined by sender +-- @provider Provider of venue database as defined by sender. Only "foursquare" need to be supported currently +-- @id Identifier of the venue in provider database as defined by sender +local function sendVenue(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, latitude, longitude, title, address, id, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageVenue", + venue_ = { + ID = "Venue", + location_ = { + ID = "Location", + latitude_ = latitude, + longitude_ = longitude + }, + title_ = title, + address_ = address, + provider_ = 'foursquare', + id_ = id + }, + }, + }, dl_cb, cmd) +end + +M.sendVenue = sendVenue + +-- User contact message +-- @contact Contact to send +-- @phone_number User's phone number +-- @first_name User first name, 1-255 characters +-- @last_name User last name +-- @user_id User identifier if known, 0 otherwise +local function sendContact(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, phone_number, first_name, last_name, user_id, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageContact", + contact_ = { + ID = "Contact", + phone_number_ = phone_number, + first_name_ = first_name, + last_name_ = last_name, + user_id_ = user_id + }, + }, + }, dl_cb, cmd) +end + +M.sendContact = sendContact + +-- Message with a game +-- @bot_user_id User identifier of a bot owned the game +-- @game_short_name Game short name +local function sendGame(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, bot_user_id, game_short_name, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageGame", + bot_user_id_ = bot_user_id, + game_short_name_ = game_short_name + }, + }, dl_cb, cmd) +end + +M.sendGame = sendGame + +-- Forwarded message +-- @from_chat_id Chat identifier of the message to forward +-- @message_id Identifier of the message to forward +local function sendForwarded(chat_id, reply_to_message_id, disable_notification, from_background, reply_markup, from_chat_id, message_id, dl_cb, cmd) + tdcli_function ({ + ID = "SendMessage", + chat_id_ = chat_id, + reply_to_message_id_ = reply_to_message_id, + disable_notification_ = disable_notification, + from_background_ = from_background, + reply_markup_ = reply_markup, + input_message_content_ = { + ID = "InputMessageForwarded", + from_chat_id_ = from_chat_id, + message_id_ = message_id + }, + }, dl_cb, cmd) +end + +M.sendForwarded = sendForwarded + +return M cli/tg/tgcli (Binary file not shown.) 0 comments on commit 3233fdf Comment on 3233fdf Leave a comment Comment Desktop version
bluss / RustfaqBluss' Rust FAQ, intended to answer questions that are often encountered, and being simple to update. There's quite a lot of editing needed to get to a useful FAQ, so we'll see if we get there.
Mdshobu / Liberty House Club Whitepaper# Liberty House Club **A Parallel Binance Chain to Enable Smart Contracts** _NOTE: This document is under development. Please check regularly for updates!_ ## Table of Contents - [Motivation](#motivation) - [Design Principles](#design-principles) - [Consensus and Validator Quorum](#consensus-and-validator-quorum) * [Proof of Staked Authority](#proof-of-staked-authority) * [Validator Quorum](#validator-quorum) * [Security and Finality](#security-and-finality) * [Reward](#reward) - [Token Economy](#token-economy) * [Native Token](#native-token) * [Other Tokens](#other-tokens) - [Cross-Chain Transfer and Communication](#cross-chain-transfer-and-communication) * [Cross-Chain Transfer](#cross-chain-transfer) * [BC to BSC Architecture](#bc-to-bsc-architecture) * [BSC to BC Architecture](#bsc-to-bc-architecture) * [Timeout and Error Handling](#timeout-and-error-handling) * [Cross-Chain User Experience](#cross-chain-user-experience) * [Cross-Chain Contract Event](#cross-chain-contract-event) - [Staking and Governance](#staking-and-governance) * [Staking on BC](#staking-on-bc) * [Rewarding](#rewarding) * [Slashing](#slashing) - [Relayers](#relayers) * [BSC Relayers](#bsc-relayers) * [Oracle Relayers](#oracle-relayers) - [Outlook](#outlook) # Motivation After its mainnet community [launch](https://www.binance.com/en/blog/327334696200323072/Binance-DEX-Launches-on-Binance-Chain-Invites-Further-Community-Development) in April 2019, [Binance Chain](https://www.binance.org) has exhibited its high speed and large throughput design. Binance Chain’s primary focus, its native [decentralized application](https://en.wikipedia.org/wiki/Decentralized_application) (“dApp”) [Binance DEX](https://www.binance.org/trade), has demonstrated its low-latency matching with large capacity headroom by handling millions of trading volume in a short time. Flexibility and usability are often in an inverse relationship with performance. The concentration on providing a convenient digital asset issuing and trading venue also brings limitations. Binance Chain's most requested feature is the programmable extendibility, or simply the [Smart Contract](https://en.wikipedia.org/wiki/Smart_contract) and Virtual Machine functions. Digital asset issuers and owners struggle to add new decentralized features for their assets or introduce any sort of community governance and activities. Despite this high demand for adding the Smart Contract feature onto Binance Chain, it is a hard decision to make. The execution of a Smart Contract may slow down the exchange function and add non-deterministic factors to trading. If that compromise could be tolerated, it might be a straightforward idea to introduce a new Virtual Machine specification based on [Tendermint](https://tendermint.com/core/), based on the current underlying consensus protocol and major [RPC](https://docs.binance.org/api-reference/node-rpc.html) implementation of Binance Chain. But all these will increase the learning requirements for all existing dApp communities, and will not be very welcomed. We propose a parallel blockchain of the current Binance Chain to retain the high performance of the native DEX blockchain and to support a friendly Smart Contract function at the same time. # Design Principles After the creation of the parallel blockchain into the Binance Chain ecosystem, two blockchains will run side by side to provide different services. The new parallel chain will be called “**Binance Smart Chain**” (short as “**BSC**” for the below sections), while the existing mainnet remains named “**Binance Chain**” (short as “**BC**” for the below sections). Here are the design principles of **BSC**: 1. **Standalone Blockchain**: technically, BSC is a standalone blockchain, instead of a layer-2 solution. Most BSC fundamental technical and business functions should be self-contained so that it can run well even if the BC stopped for a short period. 2. **Ethereum Compatibility**: The first practical and widely-used Smart Contract platform is Ethereum. To take advantage of the relatively mature applications and community, BSC chooses to be compatible with the existing Ethereum mainnet. This means most of the **dApps**, ecosystem components, and toolings will work with BSC and require zero or minimum changes; BSC node will require similar (or a bit higher) hardware specification and skills to run and operate. The implementation should leave room for BSC to catch up with further Ethereum upgrades. 3. **Staking Involved Consensus and Governance**: Staking-based consensus is more environmentally friendly and leaves more flexible option to the community governance. Expectedly, this consensus should enable better network performance over [proof-of-work](https://en.wikipedia.org/wiki/Proof_of_work) blockchain system, i.e., faster blocking time and higher transaction capacity. 4. **Native Cross-Chain Communication**: both BC and BSC will be implemented with native support for cross-chain communication among the two blockchains. The communication protocol should be bi-directional, decentralized, and trustless. It will concentrate on moving digital assets between BC and BSC, i.e., [BEP2](https://github.com/binance-chain/BEPs/blob/master/BEP2.md) tokens, and eventually, other BEP tokens introduced later. The protocol should care for the minimum of other items stored in the state of the blockchains, with only a few exceptions. # Consensus and Validator Quorum Based on the above design principles, the consensus protocol of BSC is to fulfill the following goals: 1. Blocking time should be shorter than Ethereum network, e.g. 5 seconds or even shorter. 2. It requires limited time to confirm the finality of transactions, e.g. around 1-min level or shorter. 3. There is no inflation of native token: BNB, the block reward is collected from transaction fees, and it will be paid in BNB. 4. It is compatible with Ethereum system as much as possible. 5. It allows modern [proof-of-stake](https://en.wikipedia.org/wiki/Proof_of_stake) blockchain network governance. ## Proof of Staked Authority Although Proof-of-Work (PoW) has been recognized as a practical mechanism to implement a decentralized network, it is not friendly to the environment and also requires a large size of participants to maintain the security. Ethereum and some other blockchain networks, such as [MATIC Bor](https://github.com/maticnetwork/bor), [TOMOChain](https://tomochain.com/), [GoChain](https://gochain.io/), [xDAI](https://xdai.io/), do use [Proof-of-Authority(PoA)](https://en.wikipedia.org/wiki/Proof_of_authority) or its variants in different scenarios, including both testnet and mainnet. PoA provides some defense to 51% attack, with improved efficiency and tolerance to certain levels of Byzantine players (malicious or hacked). It serves as an easy choice to pick as the fundamentals. Meanwhile, the PoA protocol is most criticized for being not as decentralized as PoW, as the validators, i.e. the nodes that take turns to produce blocks, have all the authorities and are prone to corruption and security attacks. Other blockchains, such as EOS and Lisk both, introduce different types of [Delegated Proof of Stake (DPoS)](https://en.bitcoinwiki.org/wiki/DPoS) to allow the token holders to vote and elect the validator set. It increases the decentralization and favors community governance. BSC here proposes to combine DPoS and PoA for consensus, so that: 1. Blocks are produced by a limited set of validators 2. Validators take turns to produce blocks in a PoA manner, similar to [Ethereum’s Clique](https://eips.ethereum.org/EIPS/eip-225) consensus design 3. Validator set are elected in and out based on a staking based governance ## Validator Quorum In the genesis stage, a few trusted nodes will run as the initial Validator Set. After the blocking starts, anyone can compete to join as candidates to elect as a validator. The staking status decides the top 21 most staked nodes to be the next validator set, and such an election will repeat every 24 hours. **BNB** is the token used to stake for BSC. In order to remain as compatible as Ethereum and upgradeable to future consensus protocols to be developed, BSC chooses to rely on the **BC** for staking management (Please refer to the below “[Staking and Governance](#staking-and-governance)” section). There is a **dedicated staking module for BSC on BC**. It will accept BSC staking from BNB holders and calculate the highest staked node set. Upon every UTC midnight, BC will issue a verifiable `ValidatorSetUpdate` cross-chain message to notify BSC to update its validator set. While producing further blocks, the existing BSC validators check whether there is a `ValidatorSetUpdate` message relayed onto BSC periodically. If there is, they will update the validator set after an **epoch period**, i.e. a predefined number of blocking time. For example, if BSC produces a block every 5 seconds, and the epoch period is 240 blocks, then the current validator set will check and update the validator set for the next epoch in 1200 seconds (20 minutes). ## Security and Finality Given there are more than ½\*N+1 validators are honest, PoA based networks usually work securely and properly. However, there are still cases where certain amount Byzantine validators may still manage to attack the network, e.g. through the “[Clone Attack](https://arxiv.org/pdf/1902.10244.pdf)”. To secure as much as BC, BSC users are encouraged to wait until receiving blocks sealed by more than ⅔\*N+1 different validators. In that way, the BSC can be trusted at a similar security level to BC and can tolerate less than ⅓\*N Byzantine validators. With 21 validators, if the block time is 5 seconds, the ⅔\*N+1 different validator seals will need a time period of (⅔\*21+1)*5 = 75 seconds. Any critical applications for BSC may have to wait for ⅔\*N+1 to ensure a relatively secure finality. However, besides such arrangement, BSC does introduce **Slashing** logic to penalize Byzantine validators for **double signing** or **inavailability**, which will be covered in the “Staking and Governance” section later. This Slashing logic will expose the malicious validators in a very short time and make the “Clone Attack” very hard or extremely non-beneficial to execute. With this enhancement, ½\*N+1 or even fewer blocks are enough as confirmation for most transactions. ## Reward All the BSC validators in the current validator set will be rewarded with transaction **fees in BNB**. As BNB is not an inflationary token, there will be no mining rewards as what Bitcoin and Ethereum network generate, and the gas fee is the major reward for validators. As BNB is also utility tokens with other use cases, delegators and validators will still enjoy other benefits of holding BNB. The reward for validators is the fees collected from transactions in each block. Validators can decide how much to give back to the delegators who stake their BNB to them, in order to attract more staking. Every validator will take turns to produce the blocks in the same probability (if they stick to 100% liveness), thus, in the long run, all the stable validators may get a similar size of the reward. Meanwhile, the stakes on each validator may be different, so this brings a counter-intuitive situation that more users trust and delegate to one validator, they potentially get less reward. So rational delegators will tend to delegate to the one with fewer stakes as long as the validator is still trustful (insecure validator may bring slashable risk). In the end, the stakes on all the validators will have less variation. This will actually prevent the stake concentration and “winner wins forever” problem seen on some other networks. Some parts of the gas fee will also be rewarded to relayers for Cross-Chain communication. Please refer to the “[Relayers](#relayers)” section below. # Token Economy BC and BSC share the same token universe for BNB and BEP2 tokens. This defines: 1. The same token can circulate on both networks, and flow between them bi-directionally via a cross-chain communication mechanism. 2. The total circulation of the same token should be managed across the two networks, i.e. the total effective supply of a token should be the sum of the token’s total effective supply on both BSC and BC. 3. The tokens can be initially created on BSC in a similar format as ERC20 token standard, or on BC as a BEP2, then created on the other. There are native ways on both networks to link the two and secure the total supply of the token. ## Native Token BNB will run on BSC in the same way as ETH runs on Ethereum so that it remains as “native token” for both BSC and BC. This means, in addition to BNB is used to pay most of the fees on Binance Chain and Binance DEX, BNB will be also used to: 1. pay “fees“ to deploy smart contracts on BSC 2. stake on selected BSC validators, and get corresponding rewards 3. perform cross-chain operations, such as transfer token assets across BC and BSC ### Seed Fund Certain amounts of BNB will be burnt on BC and minted on BSC during its genesis stage. This amount is called “Seed Fund” to circulate on BSC after the first block, which will be dispatched to the initial BC-to-BSC Relayer(described in later sections) and initial validator set introduced at genesis. These BNBs are used to pay transaction fees in the early stage to transfer more BNB from BC onto BSC via the cross-chain mechanism. The BNB cross-chain transfer is discussed in a later section, but for BC to BSC transfer, it is generally to lock BNB on BC from the source address of the transfer to a system-controlled address and unlock the corresponding amount from special contract to the target address of the transfer on BSC, or reversely, when transferring from BSC to BC, it is to lock BNB from the source address on BSC into a special contract and release locked amount on BC from the system address to the target address. The logic is related to native code on BC and a series of smart contracts on BSC. ## Other Tokens BC supports BEP2 tokens and upcoming [BEP8 tokens](https://github.com/binance-chain/BEPs/pull/69), which are native assets transferrable and tradable (if listed) via fast transactions and sub-second finality. Meanwhile, as BSC is Ethereum compatible, it is natural to support ERC20 tokens on BSC, which here is called “**BEP2E**” (with the real name to be introduced by the future BEPs,it potentially covers BEP8 as well). BEP2E may be “Enhanced” by adding a few more methods to expose more information, such as token denomination, decimal precision definition and the owner address who can decide the Token Binding across the chains. BSC and BC work together to ensure that one token can circulate in both formats with confirmed total supply and be used in different use cases. ### Token Binding BEP2 tokens will be extended to host a new attribute to associate the token with a BSC BEP2E token contract, called “**Binder**”, and this process of association is called “**Token Binding**”. Token Binding can happen at any time after BEP2 and BEP2E are ready. The token owners of either BEP2 or BEP2E don’t need to bother about the Binding, until before they really want to use the tokens on different scenarios. Issuers can either create BEP2 first or BEP2E first, and they can be bound at a later time. Of course, it is encouraged for all the issuers of BEP2 and BEP2E to set the Binding up early after the issuance. A typical procedure to bind the BEP2 and BEP2E will be like the below: 1. Ensure both the BEP2 token and the BEP2E token both exist on each blockchain, with the same total supply. BEP2E should have 3 more methods than typical ERC20 token standard: * symbol(): get token symbol * decimals(): get the number of the token decimal digits * owner(): get **BEP2E contract owner’s address.** This value should be initialized in the BEP2E contract constructor so that the further binding action can verify whether the action is from the BEP2E owner. 2. Decide the initial circulation on both blockchains. Suppose the total supply is *S*, and the expected initial circulating supply on BC is *K*, then the owner should lock S-K tokens to a system controlled address on BC. 3. Equivalently, *K* tokens is locked in the special contract on BSC, which handles major binding functions and is named as **TokenHub**. The issuer of the BEP2E token should lock the *K* amount of that token into TokenHub, resulting in *S-K* tokens to circulate on BSC. Thus the total circulation across 2 blockchains remains as *S*. 4. The issuer of BEP2 token sends the bind transaction on BC. Once the transaction is executed successfully after proper verification: * It transfers *S-K* tokens to a system-controlled address on BC. * A cross-chain bind request package will be created, waiting for Relayers to relay. 5. BSC Relayers will relay the cross-chain bind request package into **TokenHub** on BSC, and the corresponding request and information will be stored into the contract. 6. The contract owner and only the owner can run a special method of TokenHub contract, `ApproveBind`, to verify the binding request to mark it as a success. It will confirm: * the token has not been bound; * the binding is for the proper symbol, with proper total supply and decimal information; * the proper lock are done on both networks; 10. Once the `ApproveBind` method has succeeded, TokenHub will mark the two tokens are bounded and share the same circulation on BSC, and the status will be propagated back to BC. After this final confirmation, the BEP2E contract address and decimals will be written onto the BEP2 token as a new attribute on BC, and the tokens can be transferred across the two blockchains bidirectionally. If the ApproveBind fails, the failure event will also be propagated back to BC to release the locked tokens, and the above steps can be re-tried later. # Cross-Chain Transfer and Communication Cross-chain communication is the key foundation to allow the community to take advantage of the dual chain structure: * users are free to create any tokenization, financial products, and digital assets on BSC or BC as they wish * the items on BSC can be manually and programmingly traded and circulated in a stable, high throughput, lighting fast and friendly environment of BC * users can operate these in one UI and tooling ecosystem. ## Cross-Chain Transfer The cross-chain transfer is the key communication between the two blockchains. Essentially the logic is: 1. the `transfer-out` blockchain will lock the amount from source owner addresses into a system controlled address/contracts; 2. the `transfer-in` blockchain will unlock the amount from the system controlled address/contracts and send it to target addresses. The cross-chain transfer package message should allow the BSC Relayers and BC **Oracle Relayers** to verify: 1. Enough amount of token assets are removed from the source address and locked into a system controlled addresses/contracts on the source blockchain. And this can be confirmed on the target blockchain. 2. Proper amounts of token assets are released from a system controlled addresses/contracts and allocated into target addresses on the target blockchain. If this fails, it can be confirmed on source blockchain, so that the locked token can be released back (may deduct fees). 3. The sum of the total circulation of the token assets across the 2 blockchains are not changed after this transfer action completes, no matter if the transfer succeeds or not.  The architecture of cross-chain communication is as in the above diagram. To accommodate the 2 heteroid systems, communication handling is different in each direction. ## BC to BSC Architecture BC is a Tendermint-based, instant finality blockchain. Validators with at least ⅔\*N+1 of the total voting power will co-sign each block on the chain. So that it is practical to verify the block transactions and even the state value via **Block Header** and **Merkle Proof** verification. This has been researched and implemented as “**Light-Client Protocol**”, which are intensively discussed in [the Ethereum](https://github.com/ethereum/wiki/wiki/Light-client-protocol) community, studied and implemented for [Cosmos inter-chain communication](https://github.com/cosmos/ics/blob/a4173c91560567bdb7cc9abee8e61256fc3725e9/spec/ics-007-tendermint-client/README.md). BC-to-BSC communication will be verified in an “**on-chain light client**” implemented via BSC **Smart Contracts** (some of them may be **“pre-compiled”**). After some transactions and state change happen on BC, if a transaction is defined to trigger cross-chain communication,the Cross-chain “**package**” message will be created and **BSC Relayers** will pass and submit them onto BSC as data into the "build-in system contracts". The build-in system contracts will verify the package and execute the transactions if it passes the verification. The verification will be guaranteed with the below design: 1. BC blocking status will be synced to the light client contracts on BSC from time to time, via block header and pre-commits, for the below information: * block and app hash of BC that are signed by validators * current validatorset, and validator set update 2. the key-value from the blockchain state will be verified based on the Merkle Proof and information from above #1. After confirming the key-value is accurate and trustful, the build-in system contracts will execute the actions corresponding to the cross-chain packages. Some examples of such packages that can be created for BC-to-BSC are: 1. Bind: bind the BEP2 tokens and BEP2E 2. Transfer: transfer tokens after binding, this means the circulation will decrease (be locked) from BC and appear in the target address balance on BSC 3. Error Handling: to handle any timeout/failure event for BSC-to-BC communication 4. Validatorset update of BSC To ensure no duplication, proper message sequence and timely timeout, there is a “Channel” concept introduced on BC to manage any types of the communication. For relayers, please also refer to the below “Relayers” section. ## BSC to BC Architecture BSC uses Proof of Staked Authority consensus protocol, which has a chance to fork and requires confirmation of more blocks. One block only has the signature of one validator, so that it is not easy to rely on one block to verify data from BSC. To take full advantage of validator quorum of BC, an idea similar to many [Bridge ](https://github.com/poanetwork/poa-bridge)or Oracle blockchains is adopted: 1. The cross-chain communication requests from BSC will be submitted and executed onto BSC as transactions. The execution of the transanction wil emit `Events`, and such events can be observed and packaged in certain “**Oracle**” onto BC. Instead of Block Headers, Hash and Merkle Proof, this type of “Oracle” package directly contains the cross-chain information for actions, such as sender, receiver and amount for transfer. 2. To ensure the security of the Oracle, the validators of BC will form anothe quorum of “**Oracle Relayers**”. Each validator of the BC should run a **dedicated process** as the Oracle Relayer. These Oracle Relayers will submit and vote for the cross-chain communication package, like Oracle, onto BC, using the same validator keys. Any package signed by more than ⅔\*N+1 Oracle Relayers’ voting power is as secure as any block signed by ⅔\*N+1 of the same quorum of validators’ voting power. By using the same validator quorum, it saves the light client code on BC and continuous block updates onto BC. Such Oracles also have Oracle IDs and types, to ensure sequencing and proper error handling. ## Timeout and Error Handling There are scenarios that the cross-chain communication fails. For example, the relayed package cannot be executed on BSC due to some coding bug in the contracts. **Timeout and error handling logics are** used in such scenarios. For the recognizable user and system errors or any expected exceptions, the two networks should heal themselves. For example, when BC to BSC transfer fails, BSC will issue a failure event and Oracle Relayers will execute a refund on BC; when BSC to BC transfer fails, BC will issue a refund package for Relayer to relay in order to unlock the fund. However, unexpected error or exception may still happen on any step of the cross-chain communication. In such a case, the Relayers and Oracle Relayers will discover that the corresponding cross-chain channel is stuck in a particular sequence. After a Timeout period, the Relayers and Oracle Relayers can request a “SkipSequence” transaction, the stuck sequence will be marked as “Unexecutable”. A corresponding alerts will be raised, and the community has to discuss how to handle this scenario, e.g. payback via the sponsor of the validators, or event clear the fund during next network upgrade. ## Cross-Chain User Experience Ideally, users expect to use two parallel chains in the same way as they use one single chain. It requires more aggregated transaction types to be added onto the cross-chain communication to enable this, which will add great complexity, tight coupling, and maintenance burden. Here BC and BSC only implement the basic operations to enable the value flow in the initial launch and leave most of the user experience work to client side UI, such as wallets. E.g. a great wallet may allow users to sell a token directly from BSC onto BC’s DEX order book, in a secure way. ## Cross-Chain Contract Event Cross-Chain Contract Event (CCCE) is designed to allow a smart contract to trigger cross-chain transactions, directly through the contract code. This becomes possible based on: 1. Standard system contracts can be provided to serve operations callable by general smart contracts; 2. Standard events can be emitted by the standard contracts; 3. Oracle Relayers can capture the standard events, and trigger the corresponding cross-chain operations; 4. Dedicated, code-managed address (account) can be created on BC and accessed by the contracts on the BSC, here it is named as **“Contract Address on BC” (CAoB)**. Several standard operations are implemented: 1. BSC to BC transfer: this is implemented in the same way as normal BSC to BC transfer, by only triggered via standard contract. The fund can be transferred to any addresses on BC, including the corresponding CAoB of the transfer originating contract. 2. Transfer on BC: this is implemented as a special cross-chain transfer, while the real transfer is from **CAoB** to any other address (even another CAoB). 3. BC to BSC transfer: this is implemented as two-pass cross-chain communication. The first is triggered by the BSC contract and propagated onto BC, and then in the second pass, BC will start a normal BC to BSC cross-chain transfer, from **CAoB** to contract address on BSC. A special note should be paid on that the BSC contract only increases balance upon any transfer coming in on the second pass, and the error handling in the second pass is the same as the normal BC to BSC transfer. 4. IOC (Immediate-Or-Cancel) Trade Out: the primary goal of transferring assets to BC is to trade. This event will instruct to trade a certain amount of an asset in CAoB into another asset as much as possible and transfer out all the results, i.e. the left the source and the traded target tokens of the trade, back to BSC. BC will handle such relayed events by sending an “Immediate-Or-Cancel”, i.e. IOC order onto the trading pairs, once the next matching finishes, the result will be relayed back to BSC, which can be in either one or two assets. 5. Auction Trade Out: Such event will instruct BC to send an auction order to trade a certain amount of an asset in **CAoB** into another asset as much as possible and transfer out all the results back to BSC at the end of the auction. Auction function is upcoming on BC. There are some details for the Trade Out: 1. both can have a limit price (absolute or relative) for the trade; 2. the end result will be written as cross-chain packages to relay back to BSC; 3. cross-chain communication fees may be charged from the asset transferred back to BSC; 4. BSC contract maintains a mirror of the balance and outstanding orders on CAoB. No matter what error happens during the Trade Out, the final status will be propagated back to the originating contract and clear its internal state. With the above features, it simply adds the cross-chain transfer and exchange functions with high liquidity onto all the smart contracts on BSC. It will greatly add the application scenarios on Smart Contract and dApps, and make 1 chain +1 chain > 2 chains. # Staking and Governance Proof of Staked Authority brings in decentralization and community involvement. Its core logic can be summarized as the below. You may see similar ideas from other networks, especially Cosmos and EOS. 1. Token holders, including the validators, can put their tokens “**bonded**” into the stake. Token holders can **delegate** their tokens onto any validator or validator candidate, to expect it can become an actual validator, and later they can choose a different validator or candidate to **re-delegate** their tokens<sup>1</sup>. 2. All validator candidates will be ranked by the number of bonded tokens on them, and the top ones will become the real validators. 3. Validators can share (part of) their blocking reward with their delegators. 4. Validators can suffer from “**Slashing**”, a punishment for their bad behaviors, such as double sign and/or instability. 5. There is an “**unbonding period**” for validators and delegators so that the system makes sure the tokens remain bonded when bad behaviors are caught, the responsible will get slashed during this period. ## Staking on BC Ideally, such staking and reward logic should be built into the blockchain, and automatically executed as the blocking happens. Cosmos Hub, who shares the same Tendermint consensus and libraries with Binance Chain, works in this way. BC has been preparing to enable staking logic since the design days. On the other side, as BSC wants to remain compatible with Ethereum as much as possible, it is a great challenge and efforts to implement such logic on it. This is especially true when Ethereum itself may move into a different Proof of Stake consensus protocol in a short (or longer) time. In order to keep the compatibility and reuse the good foundation of BC, the staking logic of BSC is implemented on BC: 1. The staking token is BNB, as it is a native token on both blockchains anyway 2. The staking, i.e. token bond and delegation actions and records for BSC, happens on BC. 3. The BSC validator set is determined by its staking and delegation logic, via a staking module built on BC for BSC, and propagated every day UTC 00:00 from BC to BSC via Cross-Chain communication. 4. The reward distribution happens on BC around every day UTC 00:00. ## Rewarding Both the validator update and reward distribution happen every day around UTC 00:00. This is to save the cost of frequent staking updates and block reward distribution. This cost can be significant, as the blocking reward is collected on BSC and distributed on BC to BSC validators and delegators. (Please note BC blocking fees will remain rewarding to BC validators only.) A deliberate delay is introduced here to make sure the distribution is fair: 1. The blocking reward will not be sent to validator right away, instead, they will be distributed and accumulated on a contract; 2. Upon receiving the validator set update into BSC, it will trigger a few cross-chain transfers to transfer the reward to custody addresses on the corresponding validators. The custody addresses are owned by the system so that the reward cannot be spent until the promised distribution to delegators happens. 3. In order to make the synchronization simpler and allocate time to accommodate slashing, the reward for N day will be only distributed in N+2 days. After the delegators get the reward, the left will be transferred to validators’ own reward addresses. ## Slashing Slashing is part of the on-chain governance, to ensure the malicious or negative behaviors are punished. BSC slash can be submitted by anyone. The transaction submission requires **slash evidence** and cost fees but also brings a larger reward when it is successful. So far there are two slashable cases. ### Double Sign It is quite a serious error and very likely deliberate offense when a validator signs more than one block with the same height and parent block. The reference protocol implementation should already have logic to prevent this, so only the malicious code can trigger this. When Double Sign happens, the validator should be removed from the Validator **Set** right away. Anyone can submit a slash request on BC with the evidence of Double Sign of BSC, which should contain the 2 block headers with the same height and parent block, sealed by the offending validator. Upon receiving the evidence, if the BC verifies it to be valid: 1. The validator will be removed from validator set by an instance BSC validator set update Cross-Chain update; 2. A predefined amount of BNB would be slashed from the **self-delegated** BNB of the validator; Both validator and its delegators will not receive the staking rewards. 3. Part of the slashed BNB will allocate to the submitter’s address, which is a reward and larger than the cost of submitting slash request transaction 4. The rest of the slashed BNB will allocate to the other validators’ custody addresses, and distributed to all delegators in the same way as blocking reward. ### Inavailability The liveness of BSC relies on everyone in the Proof of Staked Authority validator set can produce blocks timely when it is their turn. Validators can miss their turn due to any reason, especially problems in their hardware, software, configuration or network. This instability of the operation will hurt the performance and introduce more indeterministic into the system. There can be an internal smart contract responsible for recording the missed blocking metrics of each validator. Once the metrics are above the predefined threshold, the blocking reward for validator will not be relayed to BC for distribution but shared with other better validators. In such a way, the poorly-operating validator should be gradually voted out of the validator set as their delegators will receive less or none reward. If the metrics remain above another higher level of threshold, the validator will be dropped from the rotation, and this will be propagated back to BC, then a predefined amount of BNB would be slashed from the **self-delegated** BNB of the validator. Both validators and delegators will not receive their staking rewards. ### Governance Parameters There are many system parameters to control the behavior of the BSC, e.g. slash amount, cross-chain transfer fees. All these parameters will be determined by BSC Validator Set together through a proposal-vote process based on their staking. Such the process will be carried on BC, and the new parameter values will be picked up by corresponding system contracts via a cross-chain communication. # Relayers Relayers are responsible to submit Cross-Chain Communication Packages between the two blockchains. Due to the heterogeneous parallel chain structure, two different types of Relayers are created. ## BSC Relayers Relayers for BC to BSC communication referred to as “**BSC Relayers**”, or just simply “Relayers”. Relayer is a standalone process that can be run by anyone, and anywhere, except that Relayers must register themselves onto BSC and deposit a certain refundable amount of BNB. Only relaying requests from the registered Relayers will be accepted by BSC. The package they relay will be verified by the on-chain light client on BSC. The successful relay needs to pass enough verification and costs gas fees on BSC, and thus there should be incentive reward to encourage the community to run Relayers. ### Incentives There are two major communication types: 1. Users triggered Operations, such as `token bind` or `cross chain transfer`. Users must pay additional fee to as relayer reward. The reward will be shared with the relayers who sync the referenced blockchain headers. Besides, the reward won't be paid the relayers' accounts directly. A reward distribution mechanism will be brought in to avoid monopolization. 2. System Synchronization, such as delivering `refund package`(caused by failures of most oracle relayers), special blockchain header synchronization(header contains BC validatorset update), BSC staking package. System reward contract will pay reward to relayers' accounts directly. If some Relayers have faster networks and better hardware, they can monopolize all the package relaying and leave no reward to others. Thus fewer participants will join for relaying, which encourages centralization and harms the efficiency and security of the network. Ideally, due to the decentralization and dynamic re-election of BSC validators, one Relayer can hardly be always the first to relay every message. But in order to avoid the monopolization further, the rewarding economy is also specially designed to minimize such chance: 1. The reward for Relayers will be only distributed in batches, and one batch will cover a number of successful relayed packages. 2. The reward a Relayer can get from a batch distribution is not linearly in proportion to their number of successful relayed packages. Instead, except the first a few relays, the more a Relayer relays during a batch period, the less reward it will collect. ## Oracle Relayers Relayers for BSC to BC communication are using the “Oracle” model, and so-called “**Oracle Relayers**”. Each of the validators must, and only the ones of the validator set, run Oracle Relayers. Each Oracle Relayer watches the blockchain state change. Once it catches Cross-Chain Communication Packages, it will submit to vote for the requests. After Oracle Relayers from ⅔ of the voting power of BC validators vote for the changes, the cross-chain actions will be performed. Oracle Replayers should wait for enough blocks to confirm the finality on BSC before submitting and voting for the cross-chain communication packages onto BC. The cross-chain fees will be distributed to BC validators together with the normal BC blocking rewards. Such oracle type relaying depends on all the validators to support. As all the votes for the cross-chain communication packages are recorded on the blockchain, it is not hard to have a metric system to assess the performance of the Oracle Relayers. The poorest performer may have their rewards clawed back via another Slashing logic introduced in the future. # Outlook It is hard to conclude for Binance Chain, as it has never stopped evolving. The dual-chain strategy is to open the gate for users to take advantage of the fast transferring and trading on one side, and flexible and extendable programming on the other side, but it will be one stop along the development of Binance Chain. Here below are the topics to look into so as to facilitate the community better for more usability and extensibility: 1. Add different digital asset model for different business use cases 2. Enable more data feed, especially DEX market data, to be communicated from Binance DEX to BSC 3. Provide interface and compatibility to integrate with Ethereum, including its further upgrade, and other blockchain 4. Improve client side experience to manage wallets and use blockchain more conveniently ------ [1]: BNB business practitioners may provide other benefits for BNB delegators, as they do now for long term BNB holders.
Adarshkumar0509 / Weather AppA weather app is a simple tool that gives real-time weather updates and forecasts for any location. It displays information like temperature, humidity, wind speed, and chances of rain, helping users plan their day easily. The app often includes features like severe weather alerts, location search, and an easy-to-use interface.
Mario-Kart-Felix / Solar Wind Hacker Book2020 was a roller coaster of major, world-shaking events. We all couldn't wait for the year to end. But just as 2020 was about to close, it pulled another fast one on us: the SolarWinds hack, one of the biggest cybersecurity breaches of the 21st century. The SolarWinds hack was a major event not because a single company was breached, but because it triggered a much larger supply chain incident that affected thousands of organizations, including the U.S. government. What is SolarWinds? SolarWinds is a major software company based in Tulsa, Okla., which provides system management tools for network and infrastructure monitoring, and other technical services to hundreds of thousands of organizations around the world. Among the company's products is an IT performance monitoring system called Orion. As an IT monitoring system, SolarWinds Orion has privileged access to IT systems to obtain log and system performance data. It is that privileged position and its wide deployment that made SolarWinds a lucrative and attractive target. What is the SolarWinds hack? The SolarWinds hack is the commonly used term to refer to the supply chain breach that involved the SolarWinds Orion system. In this hack, suspected nation-state hackers that have been identified as a group known as Nobelium by Microsoft -- and often simply referred to as the SolarWinds Hackers by other researchers -- gained access to the networks, systems and data of thousands of SolarWinds customers. The breadth of the hack is unprecedented and one of the largest, if not the largest, of its kind ever recorded. More than 30,000 public and private organizations -- including local, state and federal agencies -- use the Orion network management system to manage their IT resources. As a result, the hack compromised the data, networks and systems of thousands when SolarWinds inadvertently delivered the backdoor malware as an update to the Orion software. SolarWinds customers weren't the only ones affected. Because the hack exposed the inner workings of Orion users, the hackers could potentially gain access to the data and networks of their customers and partners as well -- enabling affected victims to grow exponentially from there. Orion Platform hack compromised networks of thousands of SolarWinds customers Hackers compromised a digitally signed SolarWinds Orion network monitoring component, opening a backdoor into the networks of thousands of SolarWinds government and enterprise customers. How did the SolarWinds hack happen? The hackers used a method known as a supply chain attack to insert malicious code into the Orion system. A supply chain attack works by targeting a third party with access to an organization's systems rather than trying to hack the networks directly. The third-party software, in this case the SolarWinds Orion Platform, creates a backdoor through which hackers can access and impersonate users and accounts of victim organizations. The malware could also access system files and blend in with legitimate SolarWinds activity without detection, even by antivirus software. SolarWinds was a perfect target for this kind of supply chain attack. Because their Orion software is used by many multinational companies and government agencies, all the hackers had to do was install the malicious code into a new batch of software distributed by SolarWinds as an update or patch. The SolarWinds hack timeline Here is a timeline of the SolarWinds hack: September 2019. Threat actors gain unauthorized access to SolarWinds network October 2019. Threat actors test initial code injection into Orion Feb. 20, 2020. Malicious code known as Sunburst injected into Orion March 26, 2020. SolarWinds unknowingly starts sending out Orion software updates with hacked code According to a U.S. Department of Homeland Security advisory, the affected versions of SolarWinds Orion are versions are 2019.4 through 2020.2.1 HF1. More than 18,000 SolarWinds customers installed the malicious updates, with the malware spreading undetected. Through this code, hackers accessed SolarWinds's customer information technology systems, which they could then use to install even more malware to spy on other companies and organizations. Who was affected? According to reports, the malware affected many companies and organizations. Even government departments such as Homeland Security, State, Commerce and Treasury were affected, as there was evidence that emails were missing from their systems. Private companies such as FireEye, Microsoft, Intel, Cisco and Deloitte also suffered from this attack. The breach was first detected by cybersecurity company FireEye. The company confirmed they had been infected with the malware when they saw the infection in customer systems. FireEye labeled the SolarWinds hack "UNC2452" and identified the backdoor used to gain access to its systems through SolarWinds as "Sunburst." Microsoft also confirmed that it found signs of the malware in its systems, as the breach was affecting its customers as well. Reports indicated Microsoft's own systems were being used to further the hacking attack, but Microsoft denied this claim to news agencies. Later, the company worked with FireEye and GoDaddy to block and isolate versions of Orion known to contain the malware to cut off hackers from customers' systems. They did so by turning the domain used by the backdoor malware used in Orion as part of the SolarWinds hack into a kill switch. The kill switch here served as a mechanism to prevent Sunburst from operating further. Nonetheless, even with the kill switch in place, the hack is still ongoing. Investigators have a lot of data to look through, as many companies using the Orion software aren't yet sure if they are free from the backdoor malware. It will take a long time before the full impact of the hack is known. Why did it take so long to detect the SolarWinds attack? With attackers having first gained access to the SolarWinds systems in September 2019 and the attack not being publicly discovered or reported until December 2020, attackers may well have had 14 or more months of unfettered access. The time it takes between when an attacker is able to gain access and the time an attack is actually discovered is often referred to as dwell time. According to a report released in January 2020 by security firm CrowdStrike, the average dwell time in 2019 was 95 days. Given that it took well over a year from the time the attackers first entered the SolarWinds network until the breach was discovered, the dwell time in the attack exceeded the average. The question of why it took so long to detect the SolarWinds attack has a lot to do with the sophistication of the Sunburst code and the hackers that executed the attack. "Analysis suggests that by managing the intrusion through multiple servers based in the United States and mimicking legitimate network traffic, the attackers were able to circumvent threat detection techniques employed by both SolarWinds, other private companies, and the federal government," SolarWinds said in its analysis of the attack. FireEye, which was the first firm to publicly report the attack, conducted its own analysis of the SolarWinds attack. In its report, FireEye described in detail the complex series of action that the attackers took to mask their tracks. Even before Sunburst attempts to connect out to its command-and-control server, the malware executes a number of checks to make sure no antimalware or forensic analysis tools are running. What was the purpose of the hack? The purpose of the hack remains largely unknown. Still, there are many reasons hackers would want to get into an organization's system, including having access to future product plans or employee and customer information held for ransom. It is also not yet clear what information, if any, hackers stole from government agencies. But the level of access appears to be deep and broad. There are speculations that many enterprises might be collateral damage, as the main focus of the attack was government agencies that make use of the SolarWinds IT management systems. Who was responsible for the hack? Federal investigators and cybersecurity agents believe a Russian espionage operation -- mostly likely Russia's Foreign Intelligence Service -- is behind the SolarWinds attack. The Russian government has denied any involvement in the attack, releasing a statement that said, "Malicious activities in the information space contradicts the principles of the Russian foreign policy, national interests and understanding of interstate relations." They also added that "Russia does not conduct offensive operations in the cyber domain." Contrary to experts in his administration, then-President Donald Trump hinted at around the time of the discovery of the SolarWinds hack that Chinese hackers might be behind the cybersecurity attack. However, he did not present any evidence to back up his claim. Shortly after his inauguration, President Joe Biden vowed that his administration intended to hold Russia accountable, through the launch of a full-scale intelligence assessment and review of the SolarWinds attack and those behind it. The president also created the position of deputy national security adviser for cybersecurity as part of the National Security Council. The role, held by veteran intelligence operative Anne Neuberger, is part of an overall bid by the Biden administration to refresh the federal government's approach to cybersecurity and better respond to nation-state actors. Naming the attack: What is Solorigate, Sunburst and Nobelium? The SolarWinds attack has a number of different names associated with it. While the attack is often referred to simply as the SolarWinds attack, that isn't the only name to know. Sunburst. This is the name of the actual malicious code injection that was planted by hackers into the SolarWinds Orion IT monitoring system code. Both SolarWinds and CrowdStrike generally refer to the attack as Sunburst. Solorigate. Microsoft initially dubbed the actual threat actor group behind the SolarWinds attack as Solorigate. It's a name that stuck and was adopted by other researchers as well as media. Nobelium. In March 2021, Microsoft decided that the primary designation for the threat actor behind the SolarWinds attack should actually be Nobelium -- the idea being that the group is active against multiple victims -- not just SolarWinds -- and uses more malware than just Sunburst. The China connection to the SolarWinds attack While it is suspected that the initial Sunburst code and the attack against SolarWinds and its users came from a threat actor based in Russia, other nation-state threat actors have also used SolarWinds in attacks. According to a Reuters report, suspected nation-state hackers based in China exploited SolarWinds during the same period of time the Sunburst attack occurred. The suspected China-based threat actors targeted the National Finance Center, which is a payroll agency within the U.S. Department of Agriculture. It is suspected that the China-based attackers did not use Sunburst, but rather a different malware that SolarWinds identifies as Supernova. Why is the SolarWinds hack important? The SolarWinds supply chain attack is a global hack, as threat actors turned the Orion software into a weapon gaining access to several government systems and thousands of private systems around the world. Due to the nature of the software -- and by extension the Sunburst malware -- having access to entire networks, many government and enterprise networks and systems face the risk of significant breaches. The hack could also be the catalyst for rapid, broad change in the cybersecurity industry. Many companies and government agencies are now in the process of devising new methods to react to these types of attacks before they happen. Governments and organizations are learning that it is not enough to build a firewall and hope it protects them. They have to actively seek out vulnerabilities in their systems, and either shore them up or turn them into traps against these types of attacks. Since the hack was discovered, SolarWinds has recommended customers update their existing Orion platform. The company has released patches for the malware and other potential vulnerabilities discovered since the initial Orion attack. SolarWinds also recommended customers not able to update Orion isolate SolarWinds servers and/or change passwords for accounts that have access to those servers. The greater White House cybersecurity focus will be crucial, some industry experts have said. But organizations should consider adopting modern software-as-a-service tools for monitoring and collaboration. While the cybersecurity industry has significantly advanced in the last decade, these kinds of attacks show that there is still a long way to go to get really secure systems. The Nobelium group continues to attack targets The suspected threat actor group behind the SolarWinds attack has remained active in 2021 and hasn't stopped at just targeting SolarWinds. On May 27, 2021, Microsoft reported that Nobelium, the group allegedly behind the SolarWinds attack, infiltrated software from email marketing service Constant Contact. According to Microsoft, Nobelium targeted approximately 3,000 email accounts at more than 150 different organizations. The initial attack vector appears to be an account used by USAID. From that initial foothold, Nobelium was able to send out phishing emails in an attempt to get victims to click on a link that would deploy a backdoor Trojan designed to steal user information.
aws-samples / Aws Fargate Server Sent Events Sse Amazon Kinesis Data StreamsWhen dealing with real-time data it is often required to send that data over the internet to various sources. Various technologies have enabled this such as web sockets and long polling. Recently server-sent events (SSE) has become a popular technology to push updates to clients. Ingesting this type of data source into AWS requires a client to be running continuously to receive those events. This sample shows how to connect to an SSE endpoint using AWS Fargate. This sample creates a docker image with a simple SSE client which publishes to an Amazon Kinesis Data Streams stream, then we simply store the event data in Amazon S3.
SE-Design / FAQ.mdNetSaver Pro ======== Please scroll down if you want to ask a question, request a feature or report a bug. Frequently Asked Questions (FAQ) -------------------------------- <a name="FAQ0"></a> **(0) How do I use NetSaver Pro?** * Enable the firewall using the switch in the action bar * Allow/deny Wi-Fi/mobile internet access using the icons along the right side of the application list You can use the settings menu to change from blacklist mode (allow all in *Settings* but block unwanted applications in list) to whitelist mode (block all in *Settings* but allow favorite applications in list). * Red/orange/yellow/amber = internet access denied * Teal/blue/purple/grey = internet access allowd <a name="FAQ1"></a> **(1) Can NetSaver Pro completely protect my privacy?** No - nothing can completely protect your privacy. NetSaver Pro will do its best, but it is limited by the fact it must use the VPN service. This is the trade-off required to make a firewall which does not require root access. The firewall can only start when Android "allows" it to start, so it will not offer protection during early boot-up (although your network may not be loaded at that time). It will, however, be much better than nothing, especially if you are not rebooting often. If you want to protect yourself more, you can (at least in theory) disable Wi-Fi and mobile data before rebooting, and only enable them on reboot, after the firewall service has started (and the small key icon is visible in the status bar). Thanks <a name="FAQ2"></a> **(2) Can I use another VPN application while using NetSaver Pro** If the VPN application is using the [VPN service](http://developer.android.com/reference/android/net/VpnService.html), then no, because NetSaver Pro needs to use this service. Android allows only one application at a time to use this service. <a name="FAQ3"></a> **(3) Can I use NetSaver Pro on any Android version?** No, the minimum required Android version is 4.0 (Lollipop) because NetSaver Pro uses the [addDisallowedApplication](http://developer.android.com/reference/android/net/VpnService.Builder.html#addDisallowedApplication(java.lang.String)) method. <a name="FAQ4"></a> **(4) Will NetSaver Pro use extra battery power?** If you didn't enable IP filtering, probably not. However, the network speed graph notification will use extra battery power. This is why the notification is shown only when the screen is on. You can decrease the update frequency using the settings to reduce the battery usage. <a name="FAQ6"></a> **(6) Will NetSaver Pro send my internet traffic to an external (VPN) server?** No, depending on the mode of operation basically one of two things will happen with your internet traffic: * When IP filtering is disabled, blocked internet traffic will be routed into the local VPN which will operate as sinkhole (in effect dropping all blocked traffic) * When IP filtering is enabled, both blocked and allowed internet traffic will be routed into the local VPN and only allowed traffic will be forwarded to the intended destination (so not to a VPN server) The [Android VPN service](http://developer.android.com/reference/android/net/VpnService.html) is being used to locally route all internet traffic to NetGuard so no root is required to build a firewall application. NetSaver Pro is unlike all other no-root firewalls applications. <a name="FAQ7"></a> **(7) Why are applications without internet permission shown?** Internet permission can be granted with each application update without user consent. By showing all applications, NetGuard allows you to control internet access even *before* such an update occurs. <a name="FAQ8"></a> **(8) What do I need to enable for the Google Play™ store app to work?** You need 3 packages (applications) enabled (use search in NetGuard to find them quickly): * com.android.vending (Play store) * com.google.android.gms (Play services) * com.android.providers.downloads (Download manager) Since the Google Play™ store app has a tendency to check for updates or even download them all by itself (even if no account is associated), one can keep it in check by enabling "*Allow when device in use*" for all 3 of these packages. Click on the down arrow on the left side of an application name and check that option, but leave the network icons set to red (hence blocked).The little human icon will appear for those packages. Note that NetSaver Pro does not require any Google service to be installed. <a name="FAQ9"></a> **(9) Why is the VPN service being restarted?** The VPN service will be restarted when you turn the screen on or off and when connectivity changes (Wi-Fi, mobile) to apply the rules with the conditions '*Allow when screen is on*' and '*Block when roaming*'. See [here](http://forum.xda-developers.com/showpost.php?p=65723629&postcount=1788) for more details. <a name="FAQ10"></a> **(10) Will you provide a Tasker plug-in?** If disabling NetSaver Pro is allowed to Tasker, any application can disabled NetSaver Pro too. Allowing to disable a security application from other applications is not a good idea. <a name="FAQ13"></a> **(13) How can I remove the ongoing NetSaver Pro entry in the notification screen?** * Long click the NetSaver Pro notification * Tap the 'i' icon * Depending on your device and/or ROMs manufacturer software customizations, you can be directed to either: * the **App Info** screen and you can uncheck '*Show notifications*' and agree to the next dialog * the **App Notifications** screen and you can toggle the '*Block*' slider to on Note that, whether or not you get a dialog warning to agree upon, this operation will disable any information or warning notifications from NetSaver Pro as well, like the new application installed notification. To read about the need for the notification in the first place, see [question 24](#FAQ24). Some Android versions display an additional notification, which might include a key icon. This notification can unfortunately not be removed. <a name="FAQ14"></a> **(14) Why can't I select OK to approve the VPN connection request?** There might be another (invisible) application on top of the VPN connection request dialog. Some known (screen dimming) applications which can cause this are *Lux Brightness*, *Night Mode* and *Twilight*. To avoid this problem, at least temporary, close all applications and/or services which may be running in the background. <a name="FAQ15"></a> **(15) Why won't you support the F-Droid builds?** Because F-Droid doesn't support reproducible builds. Read [here](https://blog.torproject.org/blog/deterministic-builds-part-one-cyberwar-and-global-compromise) why this is important. Another reason is that F-Droid builds are more often than not outdated, leaving users with an old version with known bugs. <a name="FAQ16"></a> **(16) Why are some applications shown dimmed?** Disabled applications and applications without internet permission are shown dimmed. <a name="FAQ17"></a> **(17) Why is NetSaver Pro so much memory?** It isn't, NetSaver Pro doesn't allocate any memory, except a little for displaying the user interface elements. It appeared that on some Android variants the Google Play™ store app connection, using almost 150 MB and needed for in-app donations, is incorrectly attributed to NetSaver Pro instead to the Google Play™ store app. <a name="FAQ18"></a> **(18) Why can't I findNetSaver Pro in the Google Play™ store app?** NetSaver Pro requires at least Android 4.0, so it is not available in the Google Play™ store app for devices running older Android versions. <a name="FAQ19"></a> **(19) Why does aplication XYZ still have internet access?** If you block internet access for an application, there is no way around it. However, applications could access the internet through other applications. Google Play services is handling push messages for most applications for example. You can prevent this by blocking internet access for the other application as well. Note that some applications keep trying to access the internet, which is done by sending a connection request packet. This packet goes into the VPN sinkhole when internet access for the application is blocked. This packet consists of less than 100 bytes and is counted by Android as outgoing traffic and will be visible in the speed graph notification as well. <a name="FAQ20"></a> **(20) Can I Greenify/hibernate NetGuard?** No. [Greenifying](https://play.google.com/store/apps/details?id=com.oasisfeng.greenify) or otherwise hibernating NetGuard will result in rules not being applied when connectivity changes from Wi-Fi/mobile, screen on/off and roaming/not roaming. <a name="FAQ21"></a> **(21) Does doze mode affect NNetSaver Pro?** I am not sure, because the [doze mode documentation](http://developer.android.com/training/monitoring-device-state/doze-standby.html) is not clear if the [Android VPN service](http://developer.android.com/reference/android/net/VpnService.html) will be affected. To be sure you can disable battery optimizations for NetSaver Pro manually like this: ``` Android settings > Battery > three dot menu > Battery optimizations > Dropdown > All apps > NetSaver Pro> Don't optimize > Done ``` This cannot be done from the application, because according to Google NetSaver Pro is [not an application type allowed to do this](http://developer.android.com/training/monitoring-device-state/doze-standby.html#whitelisting-cases). <a name="FAQ22"></a> **(22) Can I tether / use Wi-Fi calling while using NetGuard?** Yes, but this needs to be enabled in the settings. If it works depends on your Android version, because some Android versions have a bug preventing tethering and the VPN service to work together. Some devices hibernate Wi-Fi preventing tethering to work when the screen is off. This behavior can be disabled in the Android enhanced/advanced Wi-Fi settings. <a name="FAQ24"></a> **(24) Can you remove the notification from the status bar?** Android can kill background services at any time. This can only be prevented by turning a background service into a foreground service. Android requires an ongoing notification for all foreground services to make you aware of potential battery usage (see [question 4](#FAQ4)). So, the notification cannot be removed without causing instability. However, the notification is being marked as low priority, which should result in moving it to the bottom of the list. The key icon and/or the VPN running notification, which is shown by Android and not by NetGuard, can unfortunately not be removed. The [Google documentation](http://developer.android.com/reference/android/net/VpnService.html) says: "*A system-managed notification is shown during the lifetime of a VPN connection*". <a name="FAQ25"></a> **(25) Can you add a 'select all'?** There is no need for a select all function, because you can switch from black list to white list mode using the settings. See also [question 0](#FAQ0). <a name="FAQ27"></a> **(27) How do I read the blocked traffic log?** The columns have the following meaning: 1. Time (tap on a log entry to see the date) 1. Application icon (tap on a log entry to see the application name) 1. Application UID 1. Wi-Fi / mobile connection, green=allowed, red=blocked 1. Interactive state (screen on or off) 1. Protocol (see below) and packet flags (see below) 1. Source and destination port (tap on a log entry to lookup a destination port) 1. Source and destination IPv4 or IPv6 address (tap on a log entry to lookup a destination IP address) 1. Organization name owning the IP address (need to be enabled through the menu) Protocols: * ICMP * IGMP * ESP (IPSec) * TCP * UDP * Number = one of the protocols in [this list](https://en.wikipedia.org/wiki/List_of_IP_protocol_numbers) * 4 = IPv4 * 6 = IPv6 Packet flags: * S = SYN * A = ACK * P = PSH * F = FIN * R = RST For a detailed explanation see [here](https://en.wikipedia.org/wiki/Transmission_Control_Protocol). <a name="FAQ28"></a> **(28) Why is Google connectivity services allowed internet access by default?** The Google connectivity services system application checks if the current network is really connected to the internet. This is probably done by briefly connecting to some Google server. If this is not the case, there will be an '!' in the Wi-Fi or mobile icon in the system status bar. Recent Android versions seem not to switch connectivity from mobile to Wi-Fi when the Wi-Fi network is not really connected, even though there is a connection to the Wi-Fi network (or the other way around). On Android 6.0 and later you might get a notification asking you if you want to keep this connection on or not. To prevent a bad user experience there is a predefined rule to default allow the Google connectivity services. <a name="FAQ29"></a> **(29) Why do I get 'The item you requested is not available for purchase'?** You can only purchase pro feature when you installed NetSaver Pro from the Play store. <a name="FAQ30"></a> **(30) Can I also run AFWall+ on the same device?** Unless you are just testing NetSaver Pro, there is no current reason to use them both, since they cover the same function (firewall), although with different base needs (AFWall+ needs a rooted device) and ways of doing their thing (AFWall+ uses iptables). Also you need to keep per applicaton access rules _always_ in sync, else the application will not be able to access the network, hence bringing another level of complexity when setting and assuring things work out. Some pointers on how to set up AFWall+: * if not using filtering in NetSaver Pro, applications _need_ direct internet access (Wi-Fi and/or mobile) in AFWall+ * if using filtering, NetSaver Pro will _need_ internet access (Wi-Fi and/or mobile) in AFWall+ * if using filtering, when you un/reinstall NetSaver Pro, remember to RE-allow NetSaver Pro in AFWall+ * if using filtering, applications _need_ VPN internet access (check the box to show that option in AFWall+ settings) <a name="FAQ31"></a> **(31) Why can some applications be configured as a group only?** For a lot of purposes, including network access, Android groups applications on UID and not on package/application name. Especially system applications often have the same UID, despite having a different package and application name, these are set up like this by the ROM manufacturer at build time. These applications can only be allowed/blocked access to the internet as a group. <a name="FAQ32"></a> **(32) Why is the battery/network usage of NetSaver Pro so high?** This is because Android contributes battery and network usage which is normally contributed to other applications to NetSaver Prod in IP filtering mode. The total battery usage is slightly higher when IP filtering mode is enabled. IP filtering mode is always enabled on Android version before 5.0 and optionally enabled on later Android versions. <a name="FAQ33"></a> **(33) Can you add profiles?** Profiles are inconvenient because they need to be operated manually. Conditions like '*When screen is on*' are on the other hand convenient because they work automatic. Therefore profiles will not be added, but you are welcome to propose new conditions, however they need to be generally usable to be included. As a workaround you can use the export/import function to apply specific settings in specific circumstances. <a name="FAQ34"></a> **(34) Can you add the condition 'when on foreground'?** Recent Android versions do not allow an application to query if other applications are in the foreground or background anymore, so this cannot be added. You can use the condition '*when screen is on*' instead. <a name="FAQ35"></a> **(35) Why does the VPN not start?** NetSaver Pro "asks" Android to start the local VPN service, but some Android versions contain a bug which prevents the VPN from starting (automatically). Sometimes this is caused by updating NetSaver Pro. Unfortunately this cannot be fixed from NetSaver Pro. What you can try is to restart your device and/or revoke the VPN permissions from NetSaver Pro using the Android settings. Sometimes it helps to uninstall and install NetSaver Pro again (be sure to export your settings first). <a name="FAQ36"></a> **(36) Can you add PIN or password protection?** Since turning off the VPN service using the Android settings cannot be prevented, there is little use in adding PIN or password protection. <a name="FAQ37"></a> **(37) Why are the pro features so expensive?** The right question is "*why are there so many taxes and fees*": * VAT: 25% (depending on your country) * Google fee: 30% * Income tax: 50% So, what is left for the developer is just a fraction of what you pay. Despite NetSaver Pro being *really* a lot of work, only some of the convenience and advanced features are paid, which means that NetSaver Pro is basically free to use. Also note that most free applications will appear not to be sustainable in the end, whereas NetSaver Pro is properly maintained and supported. <br />
dadoyan / U SIMonitorThe U-SIMonitor is an application for Android that performs AT-commands to the baseband modem of mobile phones. It can obtain the security credentials and sensitive information of the cellular technology, such as permanent and temporary identities, encryption keys, location of users, etc. Alternatively, it can be used to evaluate the security of mobile operators by analyzing how frequently the keys are refreshed or how often the termorary identities are updated. U-SIMonitor does not disrupt the normal operation of the phone while it is running.
Aryia-Behroziuan / NumpyQuickstart tutorial Prerequisites Before reading this tutorial you should know a bit of Python. If you would like to refresh your memory, take a look at the Python tutorial. If you wish to work the examples in this tutorial, you must also have some software installed on your computer. Please see https://scipy.org/install.html for instructions. Learner profile This tutorial is intended as a quick overview of algebra and arrays in NumPy and want to understand how n-dimensional (n>=2) arrays are represented and can be manipulated. In particular, if you don’t know how to apply common functions to n-dimensional arrays (without using for-loops), or if you want to understand axis and shape properties for n-dimensional arrays, this tutorial might be of help. Learning Objectives After this tutorial, you should be able to: Understand the difference between one-, two- and n-dimensional arrays in NumPy; Understand how to apply some linear algebra operations to n-dimensional arrays without using for-loops; Understand axis and shape properties for n-dimensional arrays. The Basics NumPy’s main object is the homogeneous multidimensional array. It is a table of elements (usually numbers), all of the same type, indexed by a tuple of non-negative integers. In NumPy dimensions are called axes. For example, the coordinates of a point in 3D space [1, 2, 1] has one axis. That axis has 3 elements in it, so we say it has a length of 3. In the example pictured below, the array has 2 axes. The first axis has a length of 2, the second axis has a length of 3. [[ 1., 0., 0.], [ 0., 1., 2.]] NumPy’s array class is called ndarray. It is also known by the alias array. Note that numpy.array is not the same as the Standard Python Library class array.array, which only handles one-dimensional arrays and offers less functionality. The more important attributes of an ndarray object are: ndarray.ndim the number of axes (dimensions) of the array. ndarray.shape the dimensions of the array. This is a tuple of integers indicating the size of the array in each dimension. For a matrix with n rows and m columns, shape will be (n,m). The length of the shape tuple is therefore the number of axes, ndim. ndarray.size the total number of elements of the array. This is equal to the product of the elements of shape. ndarray.dtype an object describing the type of the elements in the array. One can create or specify dtype’s using standard Python types. Additionally NumPy provides types of its own. numpy.int32, numpy.int16, and numpy.float64 are some examples. ndarray.itemsize the size in bytes of each element of the array. For example, an array of elements of type float64 has itemsize 8 (=64/8), while one of type complex32 has itemsize 4 (=32/8). It is equivalent to ndarray.dtype.itemsize. ndarray.data the buffer containing the actual elements of the array. Normally, we won’t need to use this attribute because we will access the elements in an array using indexing facilities. An example >>> import numpy as np a = np.arange(15).reshape(3, 5) a array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14]]) a.shape (3, 5) a.ndim 2 a.dtype.name 'int64' a.itemsize 8 a.size 15 type(a) <class 'numpy.ndarray'> b = np.array([6, 7, 8]) b array([6, 7, 8]) type(b) <class 'numpy.ndarray'> Array Creation There are several ways to create arrays. For example, you can create an array from a regular Python list or tuple using the array function. The type of the resulting array is deduced from the type of the elements in the sequences. >>> >>> import numpy as np >>> a = np.array([2,3,4]) >>> a array([2, 3, 4]) >>> a.dtype dtype('int64') >>> b = np.array([1.2, 3.5, 5.1]) >>> b.dtype dtype('float64') A frequent error consists in calling array with multiple arguments, rather than providing a single sequence as an argument. >>> >>> a = np.array(1,2,3,4) # WRONG Traceback (most recent call last): ... TypeError: array() takes from 1 to 2 positional arguments but 4 were given >>> a = np.array([1,2,3,4]) # RIGHT array transforms sequences of sequences into two-dimensional arrays, sequences of sequences of sequences into three-dimensional arrays, and so on. >>> >>> b = np.array([(1.5,2,3), (4,5,6)]) >>> b array([[1.5, 2. , 3. ], [4. , 5. , 6. ]]) The type of the array can also be explicitly specified at creation time: >>> >>> c = np.array( [ [1,2], [3,4] ], dtype=complex ) >>> c array([[1.+0.j, 2.+0.j], [3.+0.j, 4.+0.j]]) Often, the elements of an array are originally unknown, but its size is known. Hence, NumPy offers several functions to create arrays with initial placeholder content. These minimize the necessity of growing arrays, an expensive operation. The function zeros creates an array full of zeros, the function ones creates an array full of ones, and the function empty creates an array whose initial content is random and depends on the state of the memory. By default, the dtype of the created array is float64. >>> >>> np.zeros((3, 4)) array([[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]]) >>> np.ones( (2,3,4), dtype=np.int16 ) # dtype can also be specified array([[[1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]], [[1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]]], dtype=int16) >>> np.empty( (2,3) ) # uninitialized array([[ 3.73603959e-262, 6.02658058e-154, 6.55490914e-260], # may vary [ 5.30498948e-313, 3.14673309e-307, 1.00000000e+000]]) To create sequences of numbers, NumPy provides the arange function which is analogous to the Python built-in range, but returns an array. >>> >>> np.arange( 10, 30, 5 ) array([10, 15, 20, 25]) >>> np.arange( 0, 2, 0.3 ) # it accepts float arguments array([0. , 0.3, 0.6, 0.9, 1.2, 1.5, 1.8]) When arange is used with floating point arguments, it is generally not possible to predict the number of elements obtained, due to the finite floating point precision. For this reason, it is usually better to use the function linspace that receives as an argument the number of elements that we want, instead of the step: >>> >>> from numpy import pi >>> np.linspace( 0, 2, 9 ) # 9 numbers from 0 to 2 array([0. , 0.25, 0.5 , 0.75, 1. , 1.25, 1.5 , 1.75, 2. ]) >>> x = np.linspace( 0, 2*pi, 100 ) # useful to evaluate function at lots of points >>> f = np.sin(x) See also array, zeros, zeros_like, ones, ones_like, empty, empty_like, arange, linspace, numpy.random.Generator.rand, numpy.random.Generator.randn, fromfunction, fromfile Printing Arrays When you print an array, NumPy displays it in a similar way to nested lists, but with the following layout: the last axis is printed from left to right, the second-to-last is printed from top to bottom, the rest are also printed from top to bottom, with each slice separated from the next by an empty line. One-dimensional arrays are then printed as rows, bidimensionals as matrices and tridimensionals as lists of matrices. >>> >>> a = np.arange(6) # 1d array >>> print(a) [0 1 2 3 4 5] >>> >>> b = np.arange(12).reshape(4,3) # 2d array >>> print(b) [[ 0 1 2] [ 3 4 5] [ 6 7 8] [ 9 10 11]] >>> >>> c = np.arange(24).reshape(2,3,4) # 3d array >>> print(c) [[[ 0 1 2 3] [ 4 5 6 7] [ 8 9 10 11]] [[12 13 14 15] [16 17 18 19] [20 21 22 23]]] See below to get more details on reshape. If an array is too large to be printed, NumPy automatically skips the central part of the array and only prints the corners: >>> >>> print(np.arange(10000)) [ 0 1 2 ... 9997 9998 9999] >>> >>> print(np.arange(10000).reshape(100,100)) [[ 0 1 2 ... 97 98 99] [ 100 101 102 ... 197 198 199] [ 200 201 202 ... 297 298 299] ... [9700 9701 9702 ... 9797 9798 9799] [9800 9801 9802 ... 9897 9898 9899] [9900 9901 9902 ... 9997 9998 9999]] To disable this behaviour and force NumPy to print the entire array, you can change the printing options using set_printoptions. >>> >>> np.set_printoptions(threshold=sys.maxsize) # sys module should be imported Basic Operations Arithmetic operators on arrays apply elementwise. A new array is created and filled with the result. >>> >>> a = np.array( [20,30,40,50] ) >>> b = np.arange( 4 ) >>> b array([0, 1, 2, 3]) >>> c = a-b >>> c array([20, 29, 38, 47]) >>> b**2 array([0, 1, 4, 9]) >>> 10*np.sin(a) array([ 9.12945251, -9.88031624, 7.4511316 , -2.62374854]) >>> a<35 array([ True, True, False, False]) Unlike in many matrix languages, the product operator * operates elementwise in NumPy arrays. The matrix product can be performed using the @ operator (in python >=3.5) or the dot function or method: >>> >>> A = np.array( [[1,1], ... [0,1]] ) >>> B = np.array( [[2,0], ... [3,4]] ) >>> A * B # elementwise product array([[2, 0], [0, 4]]) >>> A @ B # matrix product array([[5, 4], [3, 4]]) >>> A.dot(B) # another matrix product array([[5, 4], [3, 4]]) Some operations, such as += and *=, act in place to modify an existing array rather than create a new one. >>> >>> rg = np.random.default_rng(1) # create instance of default random number generator >>> a = np.ones((2,3), dtype=int) >>> b = rg.random((2,3)) >>> a *= 3 >>> a array([[3, 3, 3], [3, 3, 3]]) >>> b += a >>> b array([[3.51182162, 3.9504637 , 3.14415961], [3.94864945, 3.31183145, 3.42332645]]) >>> a += b # b is not automatically converted to integer type Traceback (most recent call last): ... numpy.core._exceptions.UFuncTypeError: Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind' When operating with arrays of different types, the type of the resulting array corresponds to the more general or precise one (a behavior known as upcasting). >>> >>> a = np.ones(3, dtype=np.int32) >>> b = np.linspace(0,pi,3) >>> b.dtype.name 'float64' >>> c = a+b >>> c array([1. , 2.57079633, 4.14159265]) >>> c.dtype.name 'float64' >>> d = np.exp(c*1j) >>> d array([ 0.54030231+0.84147098j, -0.84147098+0.54030231j, -0.54030231-0.84147098j]) >>> d.dtype.name 'complex128' Many unary operations, such as computing the sum of all the elements in the array, are implemented as methods of the ndarray class. >>> >>> a = rg.random((2,3)) >>> a array([[0.82770259, 0.40919914, 0.54959369], [0.02755911, 0.75351311, 0.53814331]]) >>> a.sum() 3.1057109529998157 >>> a.min() 0.027559113243068367 >>> a.max() 0.8277025938204418 By default, these operations apply to the array as though it were a list of numbers, regardless of its shape. However, by specifying the axis parameter you can apply an operation along the specified axis of an array: >>> >>> b = np.arange(12).reshape(3,4) >>> b array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]]) >>> >>> b.sum(axis=0) # sum of each column array([12, 15, 18, 21]) >>> >>> b.min(axis=1) # min of each row array([0, 4, 8]) >>> >>> b.cumsum(axis=1) # cumulative sum along each row array([[ 0, 1, 3, 6], [ 4, 9, 15, 22], [ 8, 17, 27, 38]]) Universal Functions NumPy provides familiar mathematical functions such as sin, cos, and exp. In NumPy, these are called “universal functions”(ufunc). Within NumPy, these functions operate elementwise on an array, producing an array as output. >>> >>> B = np.arange(3) >>> B array([0, 1, 2]) >>> np.exp(B) array([1. , 2.71828183, 7.3890561 ]) >>> np.sqrt(B) array([0. , 1. , 1.41421356]) >>> C = np.array([2., -1., 4.]) >>> np.add(B, C) array([2., 0., 6.]) See also all, any, apply_along_axis, argmax, argmin, argsort, average, bincount, ceil, clip, conj, corrcoef, cov, cross, cumprod, cumsum, diff, dot, floor, inner, invert, lexsort, max, maximum, mean, median, min, minimum, nonzero, outer, prod, re, round, sort, std, sum, trace, transpose, var, vdot, vectorize, where Indexing, Slicing and Iterating One-dimensional arrays can be indexed, sliced and iterated over, much like lists and other Python sequences. >>> >>> a = np.arange(10)**3 >>> a array([ 0, 1, 8, 27, 64, 125, 216, 343, 512, 729]) >>> a[2] 8 >>> a[2:5] array([ 8, 27, 64]) # equivalent to a[0:6:2] = 1000; # from start to position 6, exclusive, set every 2nd element to 1000 >>> a[:6:2] = 1000 >>> a array([1000, 1, 1000, 27, 1000, 125, 216, 343, 512, 729]) >>> a[ : :-1] # reversed a array([ 729, 512, 343, 216, 125, 1000, 27, 1000, 1, 1000]) >>> for i in a: ... print(i**(1/3.)) ... 9.999999999999998 1.0 9.999999999999998 3.0 9.999999999999998 4.999999999999999 5.999999999999999 6.999999999999999 7.999999999999999 8.999999999999998 Multidimensional arrays can have one index per axis. These indices are given in a tuple separated by commas: >>> >>> def f(x,y): ... return 10*x+y ... >>> b = np.fromfunction(f,(5,4),dtype=int) >>> b array([[ 0, 1, 2, 3], [10, 11, 12, 13], [20, 21, 22, 23], [30, 31, 32, 33], [40, 41, 42, 43]]) >>> b[2,3] 23 >>> b[0:5, 1] # each row in the second column of b array([ 1, 11, 21, 31, 41]) >>> b[ : ,1] # equivalent to the previous example array([ 1, 11, 21, 31, 41]) >>> b[1:3, : ] # each column in the second and third row of b array([[10, 11, 12, 13], [20, 21, 22, 23]]) When fewer indices are provided than the number of axes, the missing indices are considered complete slices: >>> >>> b[-1] # the last row. Equivalent to b[-1,:] array([40, 41, 42, 43]) The expression within brackets in b[i] is treated as an i followed by as many instances of : as needed to represent the remaining axes. NumPy also allows you to write this using dots as b[i,...]. The dots (...) represent as many colons as needed to produce a complete indexing tuple. For example, if x is an array with 5 axes, then x[1,2,...] is equivalent to x[1,2,:,:,:], x[...,3] to x[:,:,:,:,3] and x[4,...,5,:] to x[4,:,:,5,:]. >>> >>> c = np.array( [[[ 0, 1, 2], # a 3D array (two stacked 2D arrays) ... [ 10, 12, 13]], ... [[100,101,102], ... [110,112,113]]]) >>> c.shape (2, 2, 3) >>> c[1,...] # same as c[1,:,:] or c[1] array([[100, 101, 102], [110, 112, 113]]) >>> c[...,2] # same as c[:,:,2] array([[ 2, 13], [102, 113]]) Iterating over multidimensional arrays is done with respect to the first axis: >>> >>> for row in b: ... print(row) ... [0 1 2 3] [10 11 12 13] [20 21 22 23] [30 31 32 33] [40 41 42 43] However, if one wants to perform an operation on each element in the array, one can use the flat attribute which is an iterator over all the elements of the array: >>> >>> for element in b.flat: ... print(element) ... 0 1 2 3 10 11 12 13 20 21 22 23 30 31 32 33 40 41 42 43 See also Indexing, Indexing (reference), newaxis, ndenumerate, indices Shape Manipulation Changing the shape of an array An array has a shape given by the number of elements along each axis: >>> >>> a = np.floor(10*rg.random((3,4))) >>> a array([[3., 7., 3., 4.], [1., 4., 2., 2.], [7., 2., 4., 9.]]) >>> a.shape (3, 4) The shape of an array can be changed with various commands. Note that the following three commands all return a modified array, but do not change the original array: >>> >>> a.ravel() # returns the array, flattened array([3., 7., 3., 4., 1., 4., 2., 2., 7., 2., 4., 9.]) >>> a.reshape(6,2) # returns the array with a modified shape array([[3., 7.], [3., 4.], [1., 4.], [2., 2.], [7., 2.], [4., 9.]]) >>> a.T # returns the array, transposed array([[3., 1., 7.], [7., 4., 2.], [3., 2., 4.], [4., 2., 9.]]) >>> a.T.shape (4, 3) >>> a.shape (3, 4) The order of the elements in the array resulting from ravel() is normally “C-style”, that is, the rightmost index “changes the fastest”, so the element after a[0,0] is a[0,1]. If the array is reshaped to some other shape, again the array is treated as “C-style”. NumPy normally creates arrays stored in this order, so ravel() will usually not need to copy its argument, but if the array was made by taking slices of another array or created with unusual options, it may need to be copied. The functions ravel() and reshape() can also be instructed, using an optional argument, to use FORTRAN-style arrays, in which the leftmost index changes the fastest. The reshape function returns its argument with a modified shape, whereas the ndarray.resize method modifies the array itself: >>> >>> a array([[3., 7., 3., 4.], [1., 4., 2., 2.], [7., 2., 4., 9.]]) >>> a.resize((2,6)) >>> a array([[3., 7., 3., 4., 1., 4.], [2., 2., 7., 2., 4., 9.]]) If a dimension is given as -1 in a reshaping operation, the other dimensions are automatically calculated: >>> >>> a.reshape(3,-1) array([[3., 7., 3., 4.], [1., 4., 2., 2.], [7., 2., 4., 9.]]) See also ndarray.shape, reshape, resize, ravel Stacking together different arrays Several arrays can be stacked together along different axes: >>> >>> a = np.floor(10*rg.random((2,2))) >>> a array([[9., 7.], [5., 2.]]) >>> b = np.floor(10*rg.random((2,2))) >>> b array([[1., 9.], [5., 1.]]) >>> np.vstack((a,b)) array([[9., 7.], [5., 2.], [1., 9.], [5., 1.]]) >>> np.hstack((a,b)) array([[9., 7., 1., 9.], [5., 2., 5., 1.]]) The function column_stack stacks 1D arrays as columns into a 2D array. It is equivalent to hstack only for 2D arrays: >>> >>> from numpy import newaxis >>> np.column_stack((a,b)) # with 2D arrays array([[9., 7., 1., 9.], [5., 2., 5., 1.]]) >>> a = np.array([4.,2.]) >>> b = np.array([3.,8.]) >>> np.column_stack((a,b)) # returns a 2D array array([[4., 3.], [2., 8.]]) >>> np.hstack((a,b)) # the result is different array([4., 2., 3., 8.]) >>> a[:,newaxis] # view `a` as a 2D column vector array([[4.], [2.]]) >>> np.column_stack((a[:,newaxis],b[:,newaxis])) array([[4., 3.], [2., 8.]]) >>> np.hstack((a[:,newaxis],b[:,newaxis])) # the result is the same array([[4., 3.], [2., 8.]]) On the other hand, the function row_stack is equivalent to vstack for any input arrays. In fact, row_stack is an alias for vstack: >>> >>> np.column_stack is np.hstack False >>> np.row_stack is np.vstack True In general, for arrays with more than two dimensions, hstack stacks along their second axes, vstack stacks along their first axes, and concatenate allows for an optional arguments giving the number of the axis along which the concatenation should happen. Note In complex cases, r_ and c_ are useful for creating arrays by stacking numbers along one axis. They allow the use of range literals (“:”) >>> >>> np.r_[1:4,0,4] array([1, 2, 3, 0, 4]) When used with arrays as arguments, r_ and c_ are similar to vstack and hstack in their default behavior, but allow for an optional argument giving the number of the axis along which to concatenate. See also hstack, vstack, column_stack, concatenate, c_, r_ Splitting one array into several smaller ones Using hsplit, you can split an array along its horizontal axis, either by specifying the number of equally shaped arrays to return, or by specifying the columns after which the division should occur: >>> >>> a = np.floor(10*rg.random((2,12))) >>> a array([[6., 7., 6., 9., 0., 5., 4., 0., 6., 8., 5., 2.], [8., 5., 5., 7., 1., 8., 6., 7., 1., 8., 1., 0.]]) # Split a into 3 >>> np.hsplit(a,3) [array([[6., 7., 6., 9.], [8., 5., 5., 7.]]), array([[0., 5., 4., 0.], [1., 8., 6., 7.]]), array([[6., 8., 5., 2.], [1., 8., 1., 0.]])] # Split a after the third and the fourth column >>> np.hsplit(a,(3,4)) [array([[6., 7., 6.], [8., 5., 5.]]), array([[9.], [7.]]), array([[0., 5., 4., 0., 6., 8., 5., 2.], [1., 8., 6., 7., 1., 8., 1., 0.]])] vsplit splits along the vertical axis, and array_split allows one to specify along which axis to split. Copies and Views When operating and manipulating arrays, their data is sometimes copied into a new array and sometimes not. This is often a source of confusion for beginners. There are three cases: No Copy at All Simple assignments make no copy of objects or their data. >>> >>> a = np.array([[ 0, 1, 2, 3], ... [ 4, 5, 6, 7], ... [ 8, 9, 10, 11]]) >>> b = a # no new object is created >>> b is a # a and b are two names for the same ndarray object True Python passes mutable objects as references, so function calls make no copy. >>> >>> def f(x): ... print(id(x)) ... >>> id(a) # id is a unique identifier of an object 148293216 # may vary >>> f(a) 148293216 # may vary View or Shallow Copy Different array objects can share the same data. The view method creates a new array object that looks at the same data. >>> >>> c = a.view() >>> c is a False >>> c.base is a # c is a view of the data owned by a True >>> c.flags.owndata False >>> >>> c = c.reshape((2, 6)) # a's shape doesn't change >>> a.shape (3, 4) >>> c[0, 4] = 1234 # a's data changes >>> a array([[ 0, 1, 2, 3], [1234, 5, 6, 7], [ 8, 9, 10, 11]]) Slicing an array returns a view of it: >>> >>> s = a[ : , 1:3] # spaces added for clarity; could also be written "s = a[:, 1:3]" >>> s[:] = 10 # s[:] is a view of s. Note the difference between s = 10 and s[:] = 10 >>> a array([[ 0, 10, 10, 3], [1234, 10, 10, 7], [ 8, 10, 10, 11]]) Deep Copy The copy method makes a complete copy of the array and its data. >>> >>> d = a.copy() # a new array object with new data is created >>> d is a False >>> d.base is a # d doesn't share anything with a False >>> d[0,0] = 9999 >>> a array([[ 0, 10, 10, 3], [1234, 10, 10, 7], [ 8, 10, 10, 11]]) Sometimes copy should be called after slicing if the original array is not required anymore. For example, suppose a is a huge intermediate result and the final result b only contains a small fraction of a, a deep copy should be made when constructing b with slicing: >>> >>> a = np.arange(int(1e8)) >>> b = a[:100].copy() >>> del a # the memory of ``a`` can be released. If b = a[:100] is used instead, a is referenced by b and will persist in memory even if del a is executed. Functions and Methods Overview Here is a list of some useful NumPy functions and methods names ordered in categories. See Routines for the full list. Array Creation arange, array, copy, empty, empty_like, eye, fromfile, fromfunction, identity, linspace, logspace, mgrid, ogrid, ones, ones_like, r_, zeros, zeros_like Conversions ndarray.astype, atleast_1d, atleast_2d, atleast_3d, mat Manipulations array_split, column_stack, concatenate, diagonal, dsplit, dstack, hsplit, hstack, ndarray.item, newaxis, ravel, repeat, reshape, resize, squeeze, swapaxes, take, transpose, vsplit, vstack Questions all, any, nonzero, where Ordering argmax, argmin, argsort, max, min, ptp, searchsorted, sort Operations choose, compress, cumprod, cumsum, inner, ndarray.fill, imag, prod, put, putmask, real, sum Basic Statistics cov, mean, std, var Basic Linear Algebra cross, dot, outer, linalg.svd, vdot Less Basic Broadcasting rules Broadcasting allows universal functions to deal in a meaningful way with inputs that do not have exactly the same shape. The first rule of broadcasting is that if all input arrays do not have the same number of dimensions, a “1” will be repeatedly prepended to the shapes of the smaller arrays until all the arrays have the same number of dimensions. The second rule of broadcasting ensures that arrays with a size of 1 along a particular dimension act as if they had the size of the array with the largest shape along that dimension. The value of the array element is assumed to be the same along that dimension for the “broadcast” array. After application of the broadcasting rules, the sizes of all arrays must match. More details can be found in Broadcasting. Advanced indexing and index tricks NumPy offers more indexing facilities than regular Python sequences. In addition to indexing by integers and slices, as we saw before, arrays can be indexed by arrays of integers and arrays of booleans. Indexing with Arrays of Indices >>> >>> a = np.arange(12)**2 # the first 12 square numbers >>> i = np.array([1, 1, 3, 8, 5]) # an array of indices >>> a[i] # the elements of a at the positions i array([ 1, 1, 9, 64, 25]) >>> >>> j = np.array([[3, 4], [9, 7]]) # a bidimensional array of indices >>> a[j] # the same shape as j array([[ 9, 16], [81, 49]]) When the indexed array a is multidimensional, a single array of indices refers to the first dimension of a. The following example shows this behavior by converting an image of labels into a color image using a palette. >>> >>> palette = np.array([[0, 0, 0], # black ... [255, 0, 0], # red ... [0, 255, 0], # green ... [0, 0, 255], # blue ... [255, 255, 255]]) # white >>> image = np.array([[0, 1, 2, 0], # each value corresponds to a color in the palette ... [0, 3, 4, 0]]) >>> palette[image] # the (2, 4, 3) color image array([[[ 0, 0, 0], [255, 0, 0], [ 0, 255, 0], [ 0, 0, 0]], [[ 0, 0, 0], [ 0, 0, 255], [255, 255, 255], [ 0, 0, 0]]]) We can also give indexes for more than one dimension. The arrays of indices for each dimension must have the same shape. >>> >>> a = np.arange(12).reshape(3,4) >>> a array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]]) >>> i = np.array([[0, 1], # indices for the first dim of a ... [1, 2]]) >>> j = np.array([[2, 1], # indices for the second dim ... [3, 3]]) >>> >>> a[i, j] # i and j must have equal shape array([[ 2, 5], [ 7, 11]]) >>> >>> a[i, 2] array([[ 2, 6], [ 6, 10]]) >>> >>> a[:, j] # i.e., a[ : , j] array([[[ 2, 1], [ 3, 3]], [[ 6, 5], [ 7, 7]], [[10, 9], [11, 11]]]) In Python, arr[i, j] is exactly the same as arr[(i, j)]—so we can put i and j in a tuple and then do the indexing with that. >>> >>> l = (i, j) # equivalent to a[i, j] >>> a[l] array([[ 2, 5], [ 7, 11]]) However, we can not do this by putting i and j into an array, because this array will be interpreted as indexing the first dimension of a. >>> >>> s = np.array([i, j]) # not what we want >>> a[s] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 3 is out of bounds for axis 0 with size 3 # same as a[i, j] >>> a[tuple(s)] array([[ 2, 5], [ 7, 11]]) Another common use of indexing with arrays is the search of the maximum value of time-dependent series: >>> >>> time = np.linspace(20, 145, 5) # time scale >>> data = np.sin(np.arange(20)).reshape(5,4) # 4 time-dependent series >>> time array([ 20. , 51.25, 82.5 , 113.75, 145. ]) >>> data array([[ 0. , 0.84147098, 0.90929743, 0.14112001], [-0.7568025 , -0.95892427, -0.2794155 , 0.6569866 ], [ 0.98935825, 0.41211849, -0.54402111, -0.99999021], [-0.53657292, 0.42016704, 0.99060736, 0.65028784], [-0.28790332, -0.96139749, -0.75098725, 0.14987721]]) # index of the maxima for each series >>> ind = data.argmax(axis=0) >>> ind array([2, 0, 3, 1]) # times corresponding to the maxima >>> time_max = time[ind] >>> >>> data_max = data[ind, range(data.shape[1])] # => data[ind[0],0], data[ind[1],1]... >>> time_max array([ 82.5 , 20. , 113.75, 51.25]) >>> data_max array([0.98935825, 0.84147098, 0.99060736, 0.6569866 ]) >>> np.all(data_max == data.max(axis=0)) True You can also use indexing with arrays as a target to assign to: >>> >>> a = np.arange(5) >>> a array([0, 1, 2, 3, 4]) >>> a[[1,3,4]] = 0 >>> a array([0, 0, 2, 0, 0]) However, when the list of indices contains repetitions, the assignment is done several times, leaving behind the last value: >>> >>> a = np.arange(5) >>> a[[0,0,2]]=[1,2,3] >>> a array([2, 1, 3, 3, 4]) This is reasonable enough, but watch out if you want to use Python’s += construct, as it may not do what you expect: >>> >>> a = np.arange(5) >>> a[[0,0,2]]+=1 >>> a array([1, 1, 3, 3, 4]) Even though 0 occurs twice in the list of indices, the 0th element is only incremented once. This is because Python requires “a+=1” to be equivalent to “a = a + 1”. Indexing with Boolean Arrays When we index arrays with arrays of (integer) indices we are providing the list of indices to pick. With boolean indices the approach is different; we explicitly choose which items in the array we want and which ones we don’t. The most natural way one can think of for boolean indexing is to use boolean arrays that have the same shape as the original array: >>> >>> a = np.arange(12).reshape(3,4) >>> b = a > 4 >>> b # b is a boolean with a's shape array([[False, False, False, False], [False, True, True, True], [ True, True, True, True]]) >>> a[b] # 1d array with the selected elements array([ 5, 6, 7, 8, 9, 10, 11]) This property can be very useful in assignments: >>> >>> a[b] = 0 # All elements of 'a' higher than 4 become 0 >>> a array([[0, 1, 2, 3], [4, 0, 0, 0], [0, 0, 0, 0]]) You can look at the following example to see how to use boolean indexing to generate an image of the Mandelbrot set: >>> import numpy as np import matplotlib.pyplot as plt def mandelbrot( h,w, maxit=20 ): """Returns an image of the Mandelbrot fractal of size (h,w).""" y,x = np.ogrid[ -1.4:1.4:h*1j, -2:0.8:w*1j ] c = x+y*1j z = c divtime = maxit + np.zeros(z.shape, dtype=int) for i in range(maxit): z = z**2 + c diverge = z*np.conj(z) > 2**2 # who is diverging div_now = diverge & (divtime==maxit) # who is diverging now divtime[div_now] = i # note when z[diverge] = 2 # avoid diverging too much return divtime plt.imshow(mandelbrot(400,400)) ../_images/quickstart-1.png The second way of indexing with booleans is more similar to integer indexing; for each dimension of the array we give a 1D boolean array selecting the slices we want: >>> >>> a = np.arange(12).reshape(3,4) >>> b1 = np.array([False,True,True]) # first dim selection >>> b2 = np.array([True,False,True,False]) # second dim selection >>> >>> a[b1,:] # selecting rows array([[ 4, 5, 6, 7], [ 8, 9, 10, 11]]) >>> >>> a[b1] # same thing array([[ 4, 5, 6, 7], [ 8, 9, 10, 11]]) >>> >>> a[:,b2] # selecting columns array([[ 0, 2], [ 4, 6], [ 8, 10]]) >>> >>> a[b1,b2] # a weird thing to do array([ 4, 10]) Note that the length of the 1D boolean array must coincide with the length of the dimension (or axis) you want to slice. In the previous example, b1 has length 3 (the number of rows in a), and b2 (of length 4) is suitable to index the 2nd axis (columns) of a. The ix_() function The ix_ function can be used to combine different vectors so as to obtain the result for each n-uplet. For example, if you want to compute all the a+b*c for all the triplets taken from each of the vectors a, b and c: >>> >>> a = np.array([2,3,4,5]) >>> b = np.array([8,5,4]) >>> c = np.array([5,4,6,8,3]) >>> ax,bx,cx = np.ix_(a,b,c) >>> ax array([[[2]], [[3]], [[4]], [[5]]]) >>> bx array([[[8], [5], [4]]]) >>> cx array([[[5, 4, 6, 8, 3]]]) >>> ax.shape, bx.shape, cx.shape ((4, 1, 1), (1, 3, 1), (1, 1, 5)) >>> result = ax+bx*cx >>> result array([[[42, 34, 50, 66, 26], [27, 22, 32, 42, 17], [22, 18, 26, 34, 14]], [[43, 35, 51, 67, 27], [28, 23, 33, 43, 18], [23, 19, 27, 35, 15]], [[44, 36, 52, 68, 28], [29, 24, 34, 44, 19], [24, 20, 28, 36, 16]], [[45, 37, 53, 69, 29], [30, 25, 35, 45, 20], [25, 21, 29, 37, 17]]]) >>> result[3,2,4] 17 >>> a[3]+b[2]*c[4] 17 You could also implement the reduce as follows: >>> >>> def ufunc_reduce(ufct, *vectors): ... vs = np.ix_(*vectors) ... r = ufct.identity ... for v in vs: ... r = ufct(r,v) ... return r and then use it as: >>> >>> ufunc_reduce(np.add,a,b,c) array([[[15, 14, 16, 18, 13], [12, 11, 13, 15, 10], [11, 10, 12, 14, 9]], [[16, 15, 17, 19, 14], [13, 12, 14, 16, 11], [12, 11, 13, 15, 10]], [[17, 16, 18, 20, 15], [14, 13, 15, 17, 12], [13, 12, 14, 16, 11]], [[18, 17, 19, 21, 16], [15, 14, 16, 18, 13], [14, 13, 15, 17, 12]]]) The advantage of this version of reduce compared to the normal ufunc.reduce is that it makes use of the Broadcasting Rules in order to avoid creating an argument array the size of the output times the number of vectors. Indexing with strings See Structured arrays. Linear Algebra Work in progress. Basic linear algebra to be included here. Simple Array Operations See linalg.py in numpy folder for more. >>> >>> import numpy as np >>> a = np.array([[1.0, 2.0], [3.0, 4.0]]) >>> print(a) [[1. 2.] [3. 4.]] >>> a.transpose() array([[1., 3.], [2., 4.]]) >>> np.linalg.inv(a) array([[-2. , 1. ], [ 1.5, -0.5]]) >>> u = np.eye(2) # unit 2x2 matrix; "eye" represents "I" >>> u array([[1., 0.], [0., 1.]]) >>> j = np.array([[0.0, -1.0], [1.0, 0.0]]) >>> j @ j # matrix product array([[-1., 0.], [ 0., -1.]]) >>> np.trace(u) # trace 2.0 >>> y = np.array([[5.], [7.]]) >>> np.linalg.solve(a, y) array([[-3.], [ 4.]]) >>> np.linalg.eig(j) (array([0.+1.j, 0.-1.j]), array([[0.70710678+0.j , 0.70710678-0.j ], [0. -0.70710678j, 0. +0.70710678j]])) Parameters: square matrix Returns The eigenvalues, each repeated according to its multiplicity. The normalized (unit "length") eigenvectors, such that the column ``v[:,i]`` is the eigenvector corresponding to the eigenvalue ``w[i]`` . Tricks and Tips Here we give a list of short and useful tips. “Automatic” Reshaping To change the dimensions of an array, you can omit one of the sizes which will then be deduced automatically: >>> >>> a = np.arange(30) >>> b = a.reshape((2, -1, 3)) # -1 means "whatever is needed" >>> b.shape (2, 5, 3) >>> b array([[[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11], [12, 13, 14]], [[15, 16, 17], [18, 19, 20], [21, 22, 23], [24, 25, 26], [27, 28, 29]]]) Vector Stacking How do we construct a 2D array from a list of equally-sized row vectors? In MATLAB this is quite easy: if x and y are two vectors of the same length you only need do m=[x;y]. In NumPy this works via the functions column_stack, dstack, hstack and vstack, depending on the dimension in which the stacking is to be done. For example: >>> >>> x = np.arange(0,10,2) >>> y = np.arange(5) >>> m = np.vstack([x,y]) >>> m array([[0, 2, 4, 6, 8], [0, 1, 2, 3, 4]]) >>> xy = np.hstack([x,y]) >>> xy array([0, 2, 4, 6, 8, 0, 1, 2, 3, 4]) The logic behind those functions in more than two dimensions can be strange. See also NumPy for Matlab users Histograms The NumPy histogram function applied to an array returns a pair of vectors: the histogram of the array and a vector of the bin edges. Beware: matplotlib also has a function to build histograms (called hist, as in Matlab) that differs from the one in NumPy. The main difference is that pylab.hist plots the histogram automatically, while numpy.histogram only generates the data. >>> import numpy as np rg = np.random.default_rng(1) import matplotlib.pyplot as plt # Build a vector of 10000 normal deviates with variance 0.5^2 and mean 2 mu, sigma = 2, 0.5 v = rg.normal(mu,sigma,10000) # Plot a normalized histogram with 50 bins plt.hist(v, bins=50, density=1) # matplotlib version (plot) # Compute the histogram with numpy and then plot it (n, bins) = np.histogram(v, bins=50, density=True) # NumPy version (no plot) plt.plot(.5*(bins[1:]+bins[:-1]), n) ../_images/quickstart-2.png Further reading The Python tutorial NumPy Reference SciPy Tutorial SciPy Lecture Notes A matlab, R, IDL, NumPy/SciPy dictionary © Copyright 2008-2020, The SciPy community. Last updated on Jun 29, 2020. Created using Sphinx 2.4.4.
olehhrynyk / Blogs To Keep U Up2dateI am often asked which resources and blogs i am reading or following, so i prepared curated list of blogs to keep you up to date on various new technologies, trends, existing tech updates.
