David's Wiki

Python: Difference between revisions

← Older edit
 
(63 intermediate revisions by the same user not shown)
Line 58: Line 58:
print(f'Hello {name}! This is {program}')
print(f'Hello {name}! This is {program}')


print("%s %s" %('Hello','World',))
print("%s %s" % ('Hello','World',))


name = 'world'
name = 'world'
program ='python'
program ='python'
print('Hello {name}!This is{program}.'.format(name=name,program=program))
print('Hello {}! This is {}.'.format(name, program))
print('Hello {name}! This is {program}.'.format(name=name, program=program))
 
# Format to two decimal places
print(f"Accuracy: {accuracy:.02f}%")
 
# Format an int to 2 digits
print(f"Size: {size:02}%")
</syntaxhighlight>
</syntaxhighlight>


Line 68: Line 75:
Use Numpy to provide array functionality
Use Numpy to provide array functionality
====Array Indexing====
====Array Indexing====
[https://docs.scipy.org/doc/numpy/user/basics.indexing.html Scipy Reference]
[https://numpy.org/doc/stable/user/basics.indexing.html Numpy Indexing]
Numpy has very powerful indexing. See the above reference.


===Filesystem===
===Filesystem===
Line 88: Line 96:
# Note that splitext returns ("my_great_dataset.tar", ".gz")
# Note that splitext returns ("my_great_dataset.tar", ".gz")
</syntaxhighlight>
</syntaxhighlight>
If using Python >=3.4, you also have [https://docs.python.org/3/library/pathlib.html <code>pathlib</code>]
<syntaxhighlight lang="python">
from pathlib import Path
p = Path("my_folder")
# Join paths
pp = Path(p, "files.tar.gz")
pp.suffix      # returns ".gz"
pp.suffixes    # returns [".tar", ".gz"]
pp.name        # returns "files.tar.gz"
pp.parent      # returns "my_folder"
</syntaxhighlight>
;Notes
* One annoyance with <code>pathlib.Path</code> is that you need to convert things to strings manually
** This can be done with <code>str</code>, <code>.resolve()</code>, or <code>os.fspath()</code>
* [https://treyhunner.com/2019/01/no-really-pathlib-is-great/ "No really, pathlib is great" by Trey Hunger]


====List all files in a folder====
====List all files in a folder====
[https://stackoverflow.com/questions/3207219/how-do-i-list-all-files-of-a-directory Reference]
[https://stackoverflow.com/questions/3207219/how-do-i-list-all-files-of-a-directory Reference]
<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
gazeDir = "Gaze_txt_files"
gaze_directory = "Gaze_txt_files"
# List of folders in root folder
# List of folders in root folder
gazeFolders = [path.join(gazeDir, x) for x in os.listdir(gazeDir)]
gaze_folders = [path.join(gaze_directory, x) for x in os.listdir(gaze_directory)]
# List of files 2 folders down
# List of files 2 folders down
gazeFiles = [path.join(x, y) for x in gazeFolders for y in os.listdir(x)]
gaze_files = [path.join(x, y) for x in gaze_folders for y in os.listdir(x)]
</syntaxhighlight>
</syntaxhighlight>
See also glob.


====Read/Write entire text file into a list====
====Read/Write entire text file into a list====
Line 151: Line 180:
<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
import re
import re
myReg = re.compile(r'height:(\d+)cm')
my_regex = re.compile(r'height:(\d+)cm')
myMatch = myReg.match("height:33cm");
my_match = my_regex.match("height:33cm");
print(myMatch[1])
print(my_match[1])
# 33
# 33
</syntaxhighlight>
</syntaxhighlight>
;Notes
;Notes
* <code>re.match</code> will return None if there is no match
* <code>re.match</code> will return None if there is no match
Line 206: Line 236:
         'api_paste_code':source_code,  
         'api_paste_code':source_code,  
         'api_paste_format':'python'}  
         'api_paste_format':'python'}  
 
 
# sending post request and saving response as response object  
# sending post request and saving response as response object  
r = requests.post(url = API_ENDPOINT, data = data)  
r = requests.post(url = API_ENDPOINT, data = data)  
 
 
# extracting response text
# extracting response text
pastebin_url = r.text  
pastebin_url = r.text  
print("The pastebin URL is:%s"%pastebin_url)  
print("The pastebin URL is:%s"%pastebin_url)  
Line 236: Line 266:


===if main===
===if main===
[https://stackoverflow.com/questions/419163/what-does-if-name-main-do What does if __name__ == "__main__" do?]
If you are writing a script with functions you want to be included in other scripts, use <code>__name__</code> to detect if your script is being run or being imported.
If you are writing a script with functions you want to be included in other scripts, use <code>__name__</code> to detect if your script is being run or being imported.
[https://stackoverflow.com/questions/419163/what-does-if-name-main-do What does if __name__ == "__main__" do?]
<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
if __name__ == "__main__":
if __name__ == "__main__":
  # do..something..here
</syntaxhighlight>
</syntaxhighlight>


===iterators and iterables===
Iterables include lists, np arrays, tuples. 
To create an iterator, pass an iterable to the <code>iter()</code> function.
<syntaxhighlight lang="python">
my_arr = [1,2,3,4]
my_iter = iter(my_arr)
v1 = my_iter.next()
</syntaxhighlight>


===iterators and iterables===
<code>itertools</code> contains many helper functions for interacting with iterables and iterators.
iterators


====zip====
====zip====
Line 259: Line 299:


i.e. enumerate([a1,...], start=0) = [(0, a1), (1, a2), ...]
i.e. enumerate([a1,...], start=0) = [(0, a1), (1, a2), ...]
====slice====
<code>itertools.islice</code> will allow you to create a slice from an iterable
<syntaxhighlight lang="python">
from itertools import islice
import numpy as np
a = np.arange(5)
b = islice(a, 3)
list(b) # [0,1,2]
</syntaxhighlight>
===Exceptions===
See [https://docs.python.org/3/library/exceptions.html https://docs.python.org/3/library/exceptions.html]
;Raising
<syntaxhighlight lang="python">
raise ValueError("You have bad inputs")
assert 1=1, "Something is very wrong if 1!=1"
</syntaxhighlight>
;Try Catch/Except
<syntaxhighlight lang="python>
try:
  something_which_may_raise()
except AssertError as error:
  do_fallback()
  raise # Raise the previous error.
else:
  do_something_if_no_exception()
finally:
  finish_program_and_cleanup()
</syntaxhighlight>


==Classes==
==Classes==
===Static and Class methods===
See [https://realpython.com/instance-class-and-static-methods-demystified/ realpython]
<syntaxhighlight lang="python">
class MyClass:
    def method(self):
        return 'instance method called', self
    @classmethod
    def classmethod(cls):
        return 'class method called', cls
    @staticmethod
    def staticmethod():
        return 'static method called'
</syntaxhighlight>
;Notes
* That the Google Python style guide discourages use of static methods.
** Class methods should only be used to define alternative constructors (e.g. from_matrix).
==Multithreading==
===threading===
[https://docs.python.org/3/library/threading.html?highlight=threading#module-threading <code>import threading</code>]
Use <code>threading.Thread</code> to create a thread.
===concurrrency===
In Python 3.2+, [https://docs.python.org/3/library/concurrent.futures.html#module-concurrent.futures <code>concurrent.futures</code>] gives you access to thread pools.
<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
class Car:
import os
   def __init__(self):
import threading
from concurrent.futures import ThreadPoolExecutor, as_completed
 
executor = ThreadPoolExecutor(max_workers=os.cpu_count())
thread_lock = threading.Lock()
total = 0
 
def do_something(a, b):
  with thread_lock:
    total += a + b
  return total
 
my_futures = []
for i in range(5):
   future = executor.submit(do_something, 1, 2+i)
  my_futures.append(future)
 
for future in as_completed(my_futures):
  future.result()
executor.shutdown()
</syntaxhighlight>
</syntaxhighlight>
* <code>len(os.sched_getaffinity(0))</code> returns the number of threads available to the Python process.
* Starting in Python 3.5, if <code>max_workers</code> is none, it defaults to <code>5 * os.cpu_count()</code>.
** <code>os.cpu_count()</code> returns the number of logical CPUs (i.e. threads)
* <code>executor.shutdown()</code> will wait for all jobs to finish but you cannot submit any additional jobs from other threads, after calling shutdown.
* List operations are thread-safe but most other operations will require using a thread lock or semaphore.


==Data Structures==
==Data Structures==
===Tuples===
Tuples are immutable lists. This means that have fixed size and fixed elements, though elements themselves may be mutable.
In general, they perform marginally faster than lists so you should use tuples over lists when possible, especially as parameters to functions.
Typically people use tuples as structs, i.e. objects with structure such as coordinates. See [https://stackoverflow.com/questions/626759/whats-the-difference-between-lists-and-tuples StackOverflow: Difference between lists and tuples].
<syntaxhighlight lang="python">
# Tuple with one element
m_tuple = (1,)
# Tuple with multiple elements
vals = (1,2,3, "car")
# Return a tuple
def int_divide(a, b):
  return a // b, a % b
</syntaxhighlight>
===Lists===
===Lists===
The default data structure in Python is lists.<br>
The default data structure in Python is lists.<br>
Line 313: Line 461:
for k, v in my_map.items():
for k, v in my_map.items():
   print(k, v)
   print(k, v)
</syntaxhighlight>
==Numpy==
{{main | NumPy}}
See also Cupy which is a numpy interface implemented with CUDA for GPU acceleration. Large speedups can be had for big arrays.
===random===
Legacy code uses functions from <code>np.random.*</code>.
New code should initialize a rng using <code>np.random.default_rng()</code>. 
See [https://numpy.org/doc/stable/reference/random/generator.html Random Generator] for more details.
<syntaxhighlight lang="python">
import numpy as np
rng = np.random.default_rng()
# Random integer between [0, 6)
rng.integers(0, 6)
# array of 5 random integers
rng.integers(0, 6, size=5)
</syntaxhighlight>
</syntaxhighlight>


Line 351: Line 520:
# Write to file
# Write to file
with open("my_data.json", "w") as f:
with open("my_data.json", "w") as f:
   json.dump(my_data, f)
   json.dump(my_data, f, indent=2)


# Read from file
# Read from file
Line 360: Line 529:
; Notes
; Notes
* Using <code>json.dump(data, f)</code> will dump without pretty printing
* Using <code>json.dump(data, f)</code> will dump without pretty printing
** Add indent parameter for pretty printing.
==Type Annotations==
Python 3 supports adding type annotations. However it is not enforced at runtime. 
You can check types ahead of time using [https://google.github.io/pytype/ pytype].
<syntaxhighlight lang="python">
function add_two_values(a: float, b: float) -> float:
    return a + b
</syntaxhighlight>
==Images==
===Pillow (PIL)===
<code>pip install pillow</code>
<syntaxhighlight lang="python">
from PIL import Image, ImageOps
img = Image.open("my_image.png")
# Converts to int array of shape (H,W,4)
img = np.array(img)
</syntaxhighlight>
* <code>ImageOps.flip(img)</code> - Returns an image flipped across y axis
* <code>ImageOps.mirror(img)</code> - Returns an image flipped across x axis
===Bilinear Interpolation===
Coped from [https://stackoverflow.com/questions/12729228/simple-efficient-bilinear-interpolation-of-images-in-numpy-and-python https://stackoverflow.com/questions/12729228/simple-efficient-bilinear-interpolation-of-images-in-numpy-and-python]
{{ hidden | Bilinear Interpolation function |
<syntaxhighlight lang="python">
def bilinear_interpolate(im, x, y):
    """
    Basic bilinear interpolation
    :param im:
    :param x:
    :param y:
    :return:
    """
    x = np.asarray(x)
    y = np.asarray(y)
    x0 = np.floor(x).astype(int)
    x1 = x0 + 1
    y0 = np.floor(y).astype(int)
    y1 = y0 + 1
    x0 = np.clip(x0, 0, im.shape[1] - 1)
    x1 = np.clip(x1, 0, im.shape[1] - 1)
    y0 = np.clip(y0, 0, im.shape[0] - 1)
    y1 = np.clip(y1, 0, im.shape[0] - 1)
    Ia = im[y0, x0]
    Ib = im[y1, x0]
    Ic = im[y0, x1]
    Id = im[y1, x1]
    wa = (x1 - x) * (y1 - y)
    wb = (x1 - x) * (y - y0)
    wc = (x - x0) * (y1 - y)
    wd = (x - x0) * (y - y0)
   
    if len(Ia.shape) > len(wa.shape):
        wa = wa[..., np.newaxis]
        wb = wb[..., np.newaxis]
        wc = wc[..., np.newaxis]
        wd = wd[..., np.newaxis]
    return wa * Ia + wb * Ib + wc * Ic + wd * Id
</syntaxhighlight>
}}


==Libraries==
==Libraries==
===Numpy===
Other notable libraries.
 
===Matplotlib===
===Matplotlib===
{{main | Matplotlib}}
{{main | Matplotlib}}
Matplotlib is the main library used for making graphs.<br>
Matplotlib is the main library used for making graphs.<br>
Alternatively, there are also Python bindings for ggplot2<br>
[https://matplotlib.org/examples/ Examples]<br>
[https://matplotlib.org/examples/ Examples]<br>
[https://matplotlib.org/3.1.1/gallery/index.html Gallery]
[https://matplotlib.org/3.1.1/gallery/index.html Gallery]
Alternatively, there are also Python bindings for ggplot2<br>
===configargparse===
[https://pypi.org/project/ConfigArgParse/ ConfigArgParse] is the same as argparse except it allows you to use config files as args.
<syntaxhighlight lang="python">
parser = configargparse.ArgParser()
parser.add('-c', '--config', is_config_file=True, help='config file path')
# Parse all args, throw exception on unknown args.
parser.parse_args()
# Parse only known args.
parser.parse_known_args()
</syntaxhighlight>
If you want to use bools without store-true or store-false, you need to define an str2bool function:
[https://stackoverflow.com/questions/15008758/parsing-boolean-values-with-argparse Stack Overflow Answer]
{{ hidden | str2bool |
<syntaxhighlight lang="python">
def str2bool(val):
  """Converts the string value to a bool.
  Args:
    val: string representing true or false
  Returns:
    bool
  """
  if isinstance(val, bool):
    return val
  if val.lower() in ('yes', 'true', 't', 'y', '1'):
    return True
  elif val.lower() in ('no', 'false', 'f', 'n', '0'):
    return False
  else:
    raise argparse.ArgumentTypeError('Boolean value expected.')
#...
parser.add_argument("--augment",
                    type=str2bool,
                    help="Augment",
                    default=False)
</syntaxhighlight>
}}
[[Category:Programming languages]]
Retrieved from "https://wiki.davidl.me/view/Python"