Python Scripting

---

Course Outline

Introduction

• Distributions – python.org, Anaconda Python
• Shells – Python, Jupyter
• IDEs – PyCharm, Spyder, Eclipse
• Editors - Visual Studio Code
• First program - ‘Hello World!’
• Interpretation and .pyc, .pyo files
• Python data types
• type (), id (), sys.getsizeof()
• Values, Variables, Identifiers & Labels
• Python labeling system
• Object pooling
• Conversion functions
• The language which knew Infinity
• Console input, output
• Operators in Python
• Strings
  • Encoding
  • Define a string - Multiple quotes and
  • Multiple lines
  • String functions
  • String slicing - start, end & step
  • -ve indexing
• Built-in functions

Control Structures

• Execution flow of a Program
• Conditional statements
  • If, if-else, if-elif
  • Multiple if & Nested if-else
• Looping statements
  • while & while-else
  • for loop
  • range()
  • xrange() - Legacy
  • Containers, Generator & Iterator
  • For-else
  • break & continue
  • When to use for-else?
  • Tuple unpacking method
  • Converting List/Tuples of Tuples/Lists into Dictionary
  • Converting Dictionary to List of Tuples
  • Lambda introduction
  • Sorting List of Tuples and Dictionaries
  • Finding max(), min() in a Dictionary
  • Wherever you go, Dictionary follows you!
• USE CASES:
  • Counting Problem
  • Grouping Problem
  • Indexing
  • Caching
  • Keep the latest
• Counter() - simplest counting algorithm
• DefaultDict - Always has a value
• OrderedDict - Maintains order
• Dequeue - Short time memory loss
• Forzenset() – hashable set
• namedtuple() – hashable dict
• Heapq - efficient in-memory min-heap()
• Importance of Hashability
• Packing and Unpacking
  • Swapping two values
• Iterator using iter() and next()

Data Structures

• List
  • Introduction to List
  • Purpose of a List
  • Iterating through a List
  • List slicing, -ve indexing
  • List functions
  • List Operations
  • List of Lists
  • Comparing Lists
  • Homogeneous data
  • Built-in array.array()
  • numpy.array()
• Tuple
  • Introduction of Tuple
  • Tuple Slicing
  • -ve indexing
  • Iterating through a Tuple
  • List of Tuples
  • Purpose of a Tuple
  • List Vs Tuple
• List Vs Tuple
• Set
  • Introduction of set
  • How to remove duplicates in a list
  • How Set removes duplicates?
  • Set functions
• Sets are hashable but Lists are unhashable
• Set Use-Cases:
  • Removing duplicates
  • Common and uncommon items
  • Lookup table
• Dictionary (Most Popular Data Structure) Introduction of Dictionary - Associative data structure
  • Creating a Dictionary
  • Adding elements to Dictionary
  • Deleting key value pair
  • Updating / extending a Dictionary
  • Iterating through a Dictionary
  • Python Code Files
  • Importing functions from another file
  • __name __: Preventing unwanted code execution
  • Importing from a folder
  • Folders Vs Packages
  • __init__.py
  • Namespace
  • __all__
  • import *
  • Private global variables and functions
  • __builtins__
  • Recursive imports
  • Use case: Project Structure

Functions

• Purpose of a function
• Defining a function
• Calling a function
• Function parameter passing
  • Formal arguments
  • Actual arguments
  • Positional arguments
  • Keyword arguments
  • Variable arguments
  • Variable keyword arguments
  • Use case *args, **kwargs
• Function call stack
  • locals()
  • globals()
  • Stackframe
• Call-by-object-reference
  • Shallow copy - copy.copy()
  • Deep copy - copy.deepcopy()

Modules

• Data filtering
• Data Frames
  • Constructing from a dictionary with
  • values as Lists
  • Custom indexing
  • Rearranging the columns
  • ioc(), iloc(), at() & iat()
  • Assigning a column t the dataframe
  • Adding a new column
  • Deleting a column
  • Slicing
  • Indexing and Advanced indexing
  • Boolean indexing
  • Transposing
  • Sort by
• Concatenate
• Merge
• Join
• Group by Aggregation
• Data Munging
• Working with missing data
• Reading Data from CSV,
• Excel, JSON
• Writing Data t CSV, Excel, JSON, HTML
• Reading data from database and storing in
• Dataframe
• Writing dataframe t database
• Handling PDF files - tabula-py

File – IO

• Creating File
• File reading & writing
• Line by line file reading
• Writing multiple lines
• Reading JSON, XML, CSV & Excel
• Binary files
• Pickling
• Use case - Cleaning text

NumPy

• NumPy arrays
  • Double dimension arrays
  • Resizing, reshaping
  • Vector multiplication
  • Boolean filtering
  • Querying using where () function
  • Indexing
  • Slicing
  • Mean, Median, Standard deviation, Average
  • Transpose
  • Broadcasting
• NumPy matrix
  • Addition, multiplication
  • Transpose, inverse
• NumPy random module

Pandas

• Series
  • Constructing from dictionaries
• Custom index
• Data Collections
• Data Preprocessing
• Exploratory Data Analysis
• Profiling & Visualization